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ABSTRACT 


The  linesr!'  discrete  Kalmsn  filter  W3S  analyzed  usini^  a 
f reauency-domain  approach*  Process  and  measureirient  noise 
covariances  stg  shown  to  be  critical  design  parameters  which? 
together  with  the  assumed  prior  state  and  covariance 
estimates?  completely  determine  the  dain  schedule  of  the 
linear  Kalman  filter*  Several  relevant  design  techniaues  are 
illustrated  and  discussed*  The  concepts  of  smoothing  and 
sharpening  are  demonstrated*  E>(tensions  to  adaptive;/ 
non-linear?  and  non-parametric  filterinsi  are  briefly 
discussed?  as  are  applications  to  inventory  manasJement? 
estimation  of  time-varyind  mean  functions?  and  multiple 
regression ♦ 


»*'V  ■ 


TABLE  OF  CONTENTS 

I  *   INTRODUCTTON  8 

A*   BACKGROUND  3 

B*   PURPOSE  1.0 

C*   METHOD  10 

D>   LE^.'EL  OF  PRESENTATION  1.0 

E*   SUMMARY  OF  RESULTS  1.1 

II*   THEORY  13 

A*   STOCHASTIC  PROCESSES  AND  STATIONARITY  13 

B»   THE  PHILOSOPHICAL  CONCEPT  OF  STATIONARITY  14 

C»   DUALITY  OF  THE  TIME  AND  FREQUENCY  DOMAINS  1L=; 

!♦   Foi.jr.ier  Series  15 

2*       Basic  ConceF't  17 

3»   Discrete  Data  and  the  Samplind  Theorem  17 

4*   The  Discrete  Fourier  Trans  rorin  18 

D.  THE  DOOB-MEYER-FISK  DECOMPOSITION  20 

E,  DIGITAL  FILTERS  22 

1  ♦   S  o  iTi  e  C 1  a  s  s  i  r  i.  c  a  t  i  o  n  s    o  f    D  :i.  .'i^  i  t  a  I    f  i  1 1  e  r  s  2  3 

2  ♦   A }--  p  I  i  c .;;?  t  i  o  n  s  o  f  D  i  rA  i  t  a  I  F  i  1 1,  e  r^  s  2  4 

3»   Analysis  of  Di.^ital  Filters  24 

F*   DATA  ANALYSIS  AND  EXPERIMENT  DESIGN  25 

III.   THE  LINEAR  DISCRETE  KALMAN  FILTER  27 

A*   DESCRIPTION  27 

B.  THE  SCALAR  KALMAN  FILTER  31 

1*   Transient  and  Steady-State  Gain  31 

2*   Frequency  Response  32 

C.  IMPRO^^ING  THE  FILTER  36 

D»   PERFORMANCE  COMPARISON  — 37 


E»   THE  TRANSIENT  CASE  48 

E.   HIGHER  ORDER  FILTERS  50 

lU*   ESTIMATION y  SMOOTHING  AND  PREDICTION  62 

A»   ESTIMATION  62 

B»   SMOOTHING  62 

C*   A  COMPARISON  OF  TUO  SMOOTHERS 6'5 

D.   PREDICTION  69 

y»  SOME  REFINEMENTS  s-  EXTENSIONS  AND  ALTERNATIVES  75 

A.  ADAPTIVE  FILTERING  75 

B.  NON-LINEAR  FILTERING  73 

1.   Nan -Linear  Measurements  73 

2*   Non -Linear  Dynamics  79 

C*   NON-PARAMETRIC  FILTERING  82 

UI,   SOME  APPLICATIONS  36 

A*   INVENTORY  MANAGEMENT  — 36 

&♦   ESTIMATING  A  MEAN  FUNCTION  87 

C.  MULTIPLE  REGRESSION  39 

D«   SOME  DESIGN  CONSIDERATIONS  90 

!♦   Spectral  Analysis  of  the  Data  90 

2*   Frequency  Analysis  of  Proposed  Models  -  91 

3»   Adjust in<::i  the  Model  91 

4*   Testinsi  the  Model  92 

APPENDIX  A*   DERIVATIONS  93 

BIBLIOGRAPHY  97 

INITIAL  DISTRIBUTION  LIST  99 


4* 
%J  > 
6. 
7. 
8» 
9» 

10* 

11» 

1.2. 

13. 

14. 

15. 

16. 

17. 

18. 


LIST    OF    FIGURES 

(j.ibbs    PhenoiTienon    

L.inesr    Discrete    Kalman    Filter    

A  m  p  I  i  t  I.J  d  e    R  e  s  p  o  r'l  se    o  f    3  c  a  1  a  r    K  a  J.  ni  <;j  n    Filter    

Phase  Shift  of  Scalar  Kalman  Filter  

Impulse  Response  of  Scalar  Filters  

A  in  p  1  i  I',  u d e  C  o iti  p  a  r  i  s  o n  o  f  S  i  ri  si  1  e  a  n  d  ri o  i.i b  1  e  F 1 1 1  e  r 

Phase  Shift  of  Sinsile  and  Double  Filter  

Experiinental  Data  

Spectral  Analysis  of  Data  

Sindle  and  Double  Filter  Performance  

Sin?iile    and    Double    Filter    Performance    

Sindle  and  Double  Filter  Performance  

Gain  Schedule  Comparison  

Velocity  Filter  Impulse  Response  

Frewuency  Response  of  Position  Estimate  

Freauency  Response  of  Velocity  Estimate  

Velocity  Filter  Performance  

Velocity  Filter  Performance  


19.  Freauency  Response  of  Acceleration  Filter  

20.  Kalman  Smoother  Performance  

21.  Amplitude  Response  of  Kalman  and  Gaussian  Smoothers- 

22.  Filter  WeirJhts?  Kalman  and  Gaussian  Smoothers  

23.  Gaussian  Smoother  Performance  

24.  Com  p  a  r  i  s  o  n    o  f    K  a  1  in  a  n    a  ri  d  G  a  i..i  <r>  s  i  a  n  S  in  o  o  t  h  e  r  s  

25.  Comparison  of  Kalman  and  Gaussian  Smootfiers  

26 .  Adaptive  Fi Iters  


17 
09 

34 
35 
38 
39 
40 
42 
43 
44 
46 
47 
49 


54 


59 
61 
64 
67 
68 
70 
71 
72 
77 


I*   INTRODUCTION  AND  SUMMARY 


A,   BACKGROUND 

The  Kslman  filter  is  a  recursive  Bsyesian  least-sausres 
estimator  of  an  n-diinensional  system  state  vector  based  on  an 
m~di(Tiensional  measurement  vector*  The  filter  may  operate  in 
a  J-dimensional  coordinate  system  where  J:im?  Jin*  The  basic 
assumption  is  that  each  dimension  of  the  coordinate  system 
varies  according  to  a  kth  order  Gauss-Markov  process*  The 
Kalman  Filter  was  developed  in  the  early  1960 's  by  Kalman  and 
Bucy  Crefs*  1  and  211  ♦ 

The  Kalman  filter  may  be  used  to  obtain  an  optimal 
estimate  of  the  present  state?  a  prediction  of  future  states? 
and/or  smoothed  estimates  of  past  states*  The  current  state 
estimate  is  .Generally  used  to  determine  an  optimal  control 
input*  Future  state  estimates  are  used  to  determine  optimum 
present  policy*  Smoothed  past  state  estimates  are  used  for 
data  analysis  and  model  building*  Thus  the  potential  areas  of 
application  span  the  field  of  time  series  analysis* 

Applications  of  the  Kalman  filter  are  numerous  and  the 
theory  is  bein.^  continually  developed  and  extended*  An 
overview  of  the  development  of  linear  filtering  theory  and  an 
eirtensive  bibliography  may  be  found  in  Kailath  Cref*33*  A 
reasonably  clear  presentation  of  theory  and  applications  is 
contained  in  Gelb  Cref*4I!* 
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Perhaps  the  widest  and  most  successful  application  of 
Kalman  filtering  has  been  to  vehicle  tracking  and  control* 
Clark  Cref»5.1  has  written  a  particularly  lucid  description  of 
the  desidn  of  a  filter  for  an  anti-aircraft  dun  fire  control 
system  which  is  noteworthy  for  its  clarity  of  presentation  of 
the  underlyiniiS  theory*  It  is  evident  Crefs*  4  and  5  3  that 
the  desii^n  process  is  heuristic^  and  reauires  extensive 
testinsJ  and  analysis  of  candidate  filter  configurations!'  even 
when  the  process  is  well-understood  and  is  based  on  a  mature 
technology* 

The  Kalman  filter  has  also  been  appliedy  with  varyini^ 
desJrees  of  success  y  in  economic  models/  inventory  models  i-  and 
even  weather  models.  Considerable  difficulty  is  encountered 
in  model  buildin.di'  because  the  filter  desiisn  reouires  dood 
estimates  of  the  variance  and  covariance  of  noise  sources?  as 
well  as  an  accurate  state  transition  model*  A  prior  estimate 
of  system  state  and  covariance  is  also  reauiredj"  which  is 
somewhat  less  critical  because  errors  in  the  prior  estimate 
decrease  with  time*  These  parameters  are  often  difficult  to 
determine  in  highly  random  processes  of  Questionable 
stationari ty * 

The  Kalman  filter  is  derived  and  designed  almost  entirely 
within  the  time  domainy  although  Clark  Cref*53  does  refer  to 
the  concept  of  filter  bandwidth*  The  Kalman  filter  is 
essentially  a  low-pass  filter  with  a  very  wide  transition 
bands'  and  higher-order  filters  have  some  amplification  at  the 
mid  or  low-mid  freauency  rande*  In  iSeneralf  the  stop  band 
does  not  completely  attenuate   hi.'^h  freauencies*   This  allows 


the  filter   to  attGnuate  hi^h-f reauency  noise   somewhat  while 
still  retainins^  some  response  to  sudden  changes  of  state* 

B,   PURPOSE 

The  purpose  of  this  thesis  is  to  acQuaint  the  reader  with 
the  Kalman  filter?  to  show  how  the  choice  of  various  filter 
parameters  affect  its  performance»  and  to  provide  design 
insight  through  analysis  in  the  freauency  domain*  The 
approach  is  tutorials  and  the  reader  is  referred  to  some  of 
the  interesting  examples  which  may  be  found  in  the 
literature ♦ 

C*   METHOD 

The  freauency  response  of  several  simple  filter  desis^ns 
were  investigated  usina  the  Fast  Fourier  Transform  program  in 
the  APL  Library  2*  The  computer  results  were  Justified 
analytically  for  the  simplest  design?  a  scalar  sinsde-state 
filter*   Derivations  are  presented  in  appendix  A* 

D*   LEUEL  OF  PRESENTATION 

Full  understanding  of  the  theory  reauires  a  knowled.^e  of 
stochastic  processes  that  evolve  over  time*  as  well  as  an 
understandings  of  digital  sidnal  theory  in  the  freauency 
domain*  The  Fourier  transform  is  a  basic  tool*  A  full 
exposition  of   the  under lyinsi   theory  is   clearly  beyond   the 
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scope  of  this  presentation.  The  reader  is  directed  to  Larson 
and  Shubert  Zref*6'Jl  for  the  theory  of  stochastic  processes 
and  to  Hamming  Href*?]  for  the  theory  of  di.'^ital  filtering. 
As  previously  mentioned!'  Gelb  Zref*Al  and  Clark  Cref»5Il  are 
^ood  references   for  the   Kalman  filter*    Bloomfield  Cref*83 

and  Brillinsier  Cref»9II  are  also  applicable  references*  Brown 
Cref.lOJ  and  Box  and  Jenkins  Cref.llU  contain  related 
material ♦ 

There  are  few  readers  who  are  entirely  conversant  with 
both  the  freouency  domain  and  time  domain  approach  to  time 
series  analysis*  Nevertheless?  a  duality  exists  between  the 
two?  and  a  summary  of  the  theory  is  presented* 

Illustrative  examples  will  often  be  based  on  trackind 
modelsy  because  this  is  presently  the  widest  area  of 
application  of  Kalman  filters?  and  because  most  readers  will 
find  the  concepts  of  position?  velocity?  and  acceleration 
easy  to  understand.  The  concepts  are  easily  extendable  to 
other  areas*  For  example?  the  economist  may  wish  to  replace 
"velocity"  with  "trend"* 

E*   SUMMARY  OF  RESULTS 

The  steady-state  dain?  bandwidth?  and  sensitivity  of  the 
linear  discrete  Kalman  filter  are  shown  to  be  completely 
determined  by  the  choice  of  the  process  and  measurement  noise 
covariances*  Filter  performance  on  stationary  or 
nearly-stationary  data  can  be  predicted  by  comparing  the 
frequency  response   of  the   proposed  filter   with  a   spectral 
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analysis  of  the  data*  The  wide  transition  band  of  the 
amplitude  response  of  the  scalar  Kalman  filter  can  be 
sharpened  by  multiple  passes  of  the  data  through  a 
hij^her-dain  filter*  This  can  be  accomplished  simply  and 
recursively*   The  superiority  of   symmetric  smoothinsS  filters 

over  non-symmetric  filters  was  demonstrated.  When  used  as  a 
smoother  (by  usin^  both  forward  and  backward  passes)  the 
Kalman  filter  was  as  effective  as  a  non-recursive  Gaussian 
filter*  Hisiher-order  filters  were  shown  to  hawe  higher 
bandwidth  and  amplification  as  the  order  of  the  filter  was 
increased*  A  freouency  domain  approach  to  filter  design  may 
provide  additional  insight  and  enable  the  designer  to  achieve 
better  filter  performancey  particularly  when  the  system  state 
transition  model  and  noise  covariance  models  are  not  well 
understood . 
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II,   THEORY 


A*   STOCHASTIC  PROCESSES  AND  STATIONARITY 

A  continuous  stochastic  r-rocess  X(t)  is  3  Gaussian 
process  if  the  probability  densities  of  all  orders  are 
multivariate  Gaussian  densities*  It  is  a  kth  order 
Gauss-Markov  process  if  the  state  at  time  t  depends  only  on  k 
earlier  states*  If  we  should  expand  the  state  space  to  k 
states r  which  include  all  derivatives  up  to  the  (k"-l)th?  the 
future  system  state  vector  will  depend  only  on  the  present 
state*  For  example »  if  the  acceleration  of  a  vehicle  is  a 
first-order  Gauss-Markov  process y  then  the  position  of  the 
vehicle  is  a  third-order  Gauss-Markov  process.  Howeveri'  if 
our  state  space  includes  acceleration  and  velocity  as  well  as 
positions'  the  future  state  of  the  system  is  independent  of 
all  but  the  present  state*  If  the  random  acceleration  has 
zero  meant  and  variance  one  over  one  time  increment y  the 
acceleration  is  a  standard  Wiener  process  W(t)*  The 
derivative  of  the  Wiener  processy  written  dW(t)y  has  zero 
mean?  unit  variance?  and  is  called  white  Gaussian  noise y 
which  may  be  thought  of  as  a  'zero-th  order"  Gauss-Markov 
process  Cref*4Il* 

The  standard  Wiener  process  is  not  stationaryy  because 
the  variance  .^rows  linearly  with  time*  That  isy  the  estimate 
of  a   future  state   based  on  the   present  state   has  variance 
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that  is  3  linear  function  of  tinriG*  However »  the  standard 
Uiener  process  has  stationary s-  independent  increments*  That 
is?  the  variance  at  time  (til)  ^iven  the  state  at  time  (t)  is 
constant  and  independent  of  t» 

A  stochastic  process  X(t)  is  wide-sense  stationary  if  and 
only  if  it  has  a  constant  mean  function^  and  a  correlation 
function  such  that  Cref»6ll 

Rj,(t,  +  ^si-tj^f  s)  =  R  (t,  yt^)  =  R  (t^-t,  ) 

that   isy   the    correlation   function   of   the    process   is 

independent  of  an  arbitrary  time  shift  s»  A  Gaussian  process 
is  strictly  stationary  if  and  only  if  it  is  wide-sense 
stationary  Lvef ,61  * 

The  Gauss-Markov  assumption  makes  possible  the 
development  of  theory  and  appl  i cat  ions  y  because  f  in  ideneraly 
any  linear  operation  performed  on  a  Gaussian  process  results 
in  another  Gaussian  process?  and  the  Markov  property  allows 
consideration  of  only  the  present  state y  disreiSardind  all 
previous  states, 

B*   THE  PHILOSOPHICAL  CONCEPT  OF  STATIONARITY 

A  f reauency-domain  analysis  of  a  stochastic  process  is 
only  meaningful  if  the  process  is  stationary ♦  If  the  process 
were  chandini^i  over  timey  the  spectrum  would  chansJe  over  time* 
Since  the  spectrum  can  only  be  analyzed  by  means  of  data 
taken  over   timey  such  analysis   of  a   non-stationary  process 
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would  be  mean ini^l ess  ♦  However  y  if  the  process  is 
"auasi-ststionary "  ?  that  isi*  it  exhibits  stationar*:^ 
statistics  for  a  whilei'  then  under<3oes  a  change. •  then  settles 
down  to  stationaritvi  a^^ainy  the  freauency  approach  is  still 
useful?  although  inaccurate  over  the  transition  period*  As 
an  example y  consider  an  airplane  subject  to  random 
accelerations  due  to  air  turbulence*  An  appropriate  model 
midht  be  a  third-order  Gauss-Markov  process  as  loni^  as  the 
airplane  maintains  a  straight  path  or  turns  at  a  constant 
acceleration*  However?  the  pilot's  inputs  to  initiate  or 
terminate  a  maneuver  would  result  in  brief  periods  of 
non-stationarity y  and  the  model  would  perform  inadeouately 
durind  and  immediately  after  the  transition  period* 

It  may  be  ardued  that  every  practical  process  can  be 
considered  stationary  over  infinite  time*  If  the  process  is 
random?  it  represents  an  ensemble  of  possible  paths?  of  which 
any  realization  in  terms  of  real-world  data  is  only  one 
possible  path?  and  may  or  may  not  be  closely  representative 
of  the  ensemble*  When  dealing  with  reality?  we  are  often 
forced  to  assume  stationarity  in  order  to  make  analysis 
possible?  and  often  we  obtain  dood  results  even  though  we  can 
never  know  whether  or  not  the  assumption  of  stationarity  is 
really  valid* 

C.   DUALITY  OF  THE  TIME  AND  FREQUENCY  DOMAINS 

1 »   Fourier  Series 

A  very   wide  class  of   mathematical  functions   may  be 
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rerresentGd  by  the  Fourier  series  Cref«12.']  as    follows! 


d(t) 


+  ^^(a^cos    nt  f  b  sin  n 


t) 


*!»/ 


where  0  :i  b  -^    2  7r* 

Existence  and  convergence  of  this  series  representation 
reauire  only  that  <^  <  t )  be  everywhere  sini^le-valued  ?  and 
possess  3  finite  number  of  maxiiTia?  iTiinimay  and  finite 
discontinuities*  The  function  d(t)  need  not  be 
dif  f  erentiable  ♦  Any  function  meetinsi  the  above  criteria  can 
be  thoudht  of  as  a  constant  mean  function  a^r  plus  an 
infinite  series  of  sines  and  cosines  of  integral  freauencies 
and  various  amplitudes*  Of  course?  the  independent  variable 
t  must  be  shifted  and  scaled  to  the  interval  C  0 ? 2 7T 3 ♦  Note 
that  the  lowest  freauency  present?  aside  from  the 
zero-f  reauency  meany  is  one  cycle  for  the  span  of  i^  ( t )  ♦ 
Amon<3  the  functions  meetind  the  criteria  are  a  souare  pulse? 
an  impulse?  and  any  manifestation  of  a  random  walk*  In 
practice?  the  Fourier  analysis  of  a  function  s5  ( t )  reauires 
the  truncation  of  the  infinite  Fourier  series*  This  results 
in  a  smooth  least-sauares  approximation  to  the  function  <3  ( t )  ♦ 
There  are  ripples  in  the  approximation  if  the  function  ^(t) 
is  not  dif f erentiable  or  if  the  truncation  is  too  severe* 
This  is  known  as  the  Gitabs  phenomenon?  and  is  illustrated  in 
fisiure  1?  which  was  taken  from  Hamming  Cref*73*  By  taking  a 
sufficient  number  of  terms  in  the  Fourier  expansion?  we  can 
improve  the  closeness  of  the  approximation* 
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7r/4  7r/2  3  7r/4  v 

Figure    1»       The    Gibbs    Phenonrienori 


2*   Basic  Concept 

The  basic  concept  of  the  duality  between  the  time  and 
freauency  domains  is  so  simple  that  it  often  sJets  lost  in  a 
forest  of  Fourier  transforms*  The  time  period  is  the 
reciprocal  of  freauency*   The  basic  relationship  is 

v/27r  =  f  =  1/T 

where  v    is  the   frequency  in   radians/unit  timer   f  is  the 

freauency  in  cycles/unit   timer  and  T  is  the   time  period  for 

one  cycle*  Stated  simply y  freauency   is  the  inverse   of  the 
time  period* 

3*   Discrete  Data  and  the  Sampling  Theorem 


The  digital  computer  allows  the  efficient  analysis  of 
continuous  phenomena  by  means  of  discrete  approximations*  Ue 
saw  earlier  that  the  lowest  freauency  contained  in  a  Fourier 
expansion  of  a  function  s-Kt)  was  the  reciprocal  of  the  time 
span  covered  by  the  function*  Similarly?  the  famous  Samplin?.^ 
Theorem  Crefs*  6  and  7  3  states  that  if  a  function  ^(t)  in 
continuous   time   is   sampled    at   constant?   discrete   time 


17 


intervals  ht  (that  isy  at  a  rate  of  :l./t).t)y  then  the  hisihest 
ob«>ervable  freauency  is  0*5  cycles  per  measurenient  interval 
t^t*  This  means  that  at  least  two  observations  Bve  reouired  in 
each  cycle  in  order  to  observe  that  particular  freauency* 
The  fre«uency  0»5/tity  usually  written  sinfiPly  0  ♦  5  y  is 
referred  to  as  the  Nyauist  freauency*  The  result  is  the 
aliasing  phenomenon!'  which  is  familiar  to  most  movie.doers* 
Durinsi  the  chaser  the  stas^ecoach  wheels  appear  to  stop  or 
rotate  slowly  backwards  when  the  rate  of  rotation  of  the 
wheel  sPokes  (spokes/sec)  e;cceeds  1/2  the  camera  rate 
(frames/sec)*  When  hislher  freauencies  exist  in  the  function 
d(t)  sampled  at  a  rate  ls.tr  they  are  folded  back  and  appear  in 
the  frequency  spectrum  of  the  sampled  data  as  freauencies 
less  than  the  Nyauist  freouency*  The  sampling  theorem  shows 
that  a  spectral  analysis  of  discrete  data  is  only  meanin:^ful 
over  the  Nyauist  interval  C  -0»5/l!^t!'  fO»5/LtI]t 

4  ♦   The  Discrete  Fourier  Transform 

Any  function  d(t)  for  which  a  conversSent  F"ourier 
series  exists  may  be  represented  in  the  freauency  domain  in 
terms  of  real  and  imaginary  parts?  or  in  terms  of  amplitude 
and  Phase  andley  as  a  function  of  freauency*  It  should  be 
noted  that  the  function  .<3(t)  may  also  be  complex-valued  r  but 
we  will  deal  with  only  real-valued  functions*  In  the 
continuous  domain?  the  formulas 


d(t)  ~T^\G(v)    exp(ivt)  dv 


and 
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G(v)  =  r<3(t)  exp(-ivt)  dt 

represent  a  Fourier  transform  pair*  The  frequency  response 
G(v)  completely  determines  the  time  function  ^(t)  and 
conversely ♦ 

If   the   function   d ( t )   is  sampled   at   intervals   t   = 
0yl!'2y»»»yn  the  time~to~f reouency  transformation  becomes 


G(v)  =  21^<t)  e>jp(-ivt) 


t  =  09l9t**fr\ 


which  is  defined  only  on  the  Nyauist  interval  C-TTfTTj^  here 
defined  in  radians*  The  formula  may  he  written  in  a  more 
familiar  form  by  U5in<3  the  Euler  relation 

eKP(~ivt)  =  cos  vt  ~  i  sin  vt 


as 


G  (  V  )  •-  ^  £i(t)  (cos  vt  -  i  si  n  v  t ) 


which  is  continuous  in  v  on  the  interval  Z-TTr  Tf  1 ,  The  Fourier 
transform  is  a  bit  difficult  to  handle  analytically  for  all 
but  the  simplest  functions?  but  the  discrete  Fourier 
transform  is  .Generally  easy  to  compute  by  use  of  a  Fast 
Fourier  Transform  (FFT)  program  available  in  most  computer 
libraries*  The  output  will  Generally  be  a  very  close  discrete 
approximation  to  G(v)r  if  the  span  of  d(t)  is  larde  enoui^h* 
The  inverse  transformation  can  also  be  made* 
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Since  G(v)  is  coinpleM  valued  whenever  the  function  <i(t) 
is  not  symmet ric y  it  is  often  useful  to  represent  it  in  terms 
of  amplitude  and  phase*   The  amplitude  is 

I G  ( V )  I  =••  /g(v)G(-v)   :=   /i:Re(v)/  f  i::im(v)a^ 

where  Re(v)   and  Im(v)  are    the  real   and  imai^inary   parts  of 
G(v)*   The  phase  an^le  is 

6(v)  =  arctan  Im(v)  /  Re(v) 


ru   THE  DOOB-MEYER-FISK  DECOMPOSITION 

In    most   practical    app 1 i cat ions y   a    finite-variance 
sample-continuous  stochastic  process  X(t)  can  be  written 


T  T 

X(T)  ^    X(0)  f   /A(t)dt  f   /B(t)dW(t) 

e>  0 


where  X(0)  is  the  initial  value  of  that  process y  A(t)dt  is 
predictable y  smooth  behavior  determined  by  a  set  of 
deterministic  differential  e«uations  describing  the  systemy 
and  B(t)dU(t)  is  noise y  where  dU(t)  is  white  Gaussian  noisey 
and  B(t)  is  a  smooth  transformation  that  is  sometimes  thou.<.=iht 
of  as  "coloring"  the  noise*  Such  a  representation  is  called 
the  Doob-Meyer-Fisk  decomposition  Cref*6Ily  which  may  be 
thoui^ht  of  as  separatin?^  the  process  into  a  si.^nal  and  noise* 
Several  important   points  must   be  made   with  redard   to  this 
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o?Quat.ion*  It  is  not  intended  here  that  the  expression  be 
evaluated  analytical  l«j  ♦  The  integral  E<(t)dW(t)  is  an  I  to 
intei^raly  which  is  not  even  a  stochastic  version  of  a 
StieltJes  inte?iJral  Cref.6J»  Alsoy  althousih  the  processes 
A<t)  and  B(t)  are  smooth  functions  that  may  he  considered 
deterministic  representations  of  system  behaviors'  they  are> 
not  necessarily  known  to  the  observer?  even  when  an  adequate 
technolorjical  representation  exists* 

Consider  asJa in  our  piloted  aircraft  beinsJ  tracked  by  a 
radar*  The  process   A(t)   represents   the  dynamics  of       the 

airframe-;  as  affected  by  the  control  inputs  of  the  pilots 
which  arG  unknown  to  the  radar  observer*  The  process  B(t) 
consists  of  several  parts*  One  is  the  measurement  process? 
which  may  or  may  not  be  known  to  the  radar  observer*  For 
examples'  a  nutatind  radar  antenna  miidht  impose  some  periodic 
error  in  the  measurements'  which  would  be  manifested  in  the 
process  B<t)*  Overlaid  on  this  mi.<3ht  be  a  white  Gaussian 
noise  measurement  error*  Air  turbulence  could  also  be 
represented  as  white  Gaussian  noise  y  which  s'  however  y  coul'd 
only  be  manifested  throu-^h  deterministic  airplane  dynamics* 
There  are  those  who  would  ardue  that  the  pilot  should  also  be 
modelled  as  a  random  variable*  In  any  event?  the  process 
B<t)  mi<^ht  be  further  decomposed  into  several  processes?  here 
at  least  airframe  response  to  air  turbulence  and  periodic 
radar    antenna  dynamics* 

The  vital  observation  is  that  if  the  freauency  content  of 
the  processes  A(t)  and  B(t)  are  known  to  be  different?  they 
can   be  partially   separated  by   a  spectral   analysis  of   the 
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d  <3 1 3  ♦  I  n  our  G  X  3  rii  p  1  e  y  aire  r  a  f  t  h  a  v  e  ri  ^j  t  u  v^  a  1  d  y  |-|  a  in  i  c  i^  e  s  p  o  n  s  e 
freauenciss       in  all   control  axes.  These   can  be   estimated 

cloaelyi'  even  for  eneiirj  airr-lanesy  and  are  denerall<:i  similar 
amond  similar  types  of  airplanes  y  althou'^h  they  vary  with 
airspeed.  It  is  physically  impossible  for  the  airplane  to 
respond  faster  than  its  hii^hest  natural  dynamic  freauencies* 
Any  freauency  content  higher  than  this  must  be  noise*  If  the 
ratiar  system  dynamics  are  of  a  his/her  freauency  than  this? 
they  can  also  he  separated*  The  pilot  will  take  advantaj^e  of 
the  full  response  rate  of  the  airplane  only  very  rarely* 
Therefore?  low   freauency  components  are      most  likely   due  to 

pilot  maneuvers*  Of  courses'  since  white  Gaussian  noise  has 
a  flat  freauency  spectrum  as  a  result  of  aliasing  Cref*73.»  it 
is  impossible  to  separate  all  of  the  noise  from  the  sisinal* 
However!'  it  is  often  possible  to  remove  ouite  a  bit  of  it* 

E*   DIGITAL  FILTERS 

A  dii^ital  filter  is  a  linear  transformation  applied 
iteratively  to  a  set  of  data  points*  The  purpose  here  is  to 
separate  noise  from  the  signal*  The  simplest  digital  filter 
is  the  simple  a^'/erai^er  which  estimates  the  mean  value  from 
the  datay  and  smooths  out  all  fluctuations*  The  most  sJeneral 
form  of    the  dis^ital  filter  was  stated  by  Hamming  Cref*73  as 


M(t)  =   2a(k)  z(t~k)  -f  21b(k)  yAt-k) 


where   the   estimate   x<t)   at   some   point   t   is   a   linear 
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combination  of  the  data  points  -zit-k)  9  and  perhaps  of  the 
previous  estimates  x'(t~k)»  The  coefficients  sCk)  and  b(k) 
avQ  wei.'3htind  coefficients  and  may?  of  course )»  be  zero*  As  a 
result  of  the  samplins{  theoremy  the  filtering  process  is 
meaningless  unless  the  measurements  z(t~k)  are  made  at 
eaually  spaced  intervals  alon^  the  t  axisr  where  t  is 
usually?  but  not  necessarily y  time» 

1  ♦   Some  Classifications  of  Di.^^ital  Filters 

Didital  filters  may  be  classified  as  symmetric  or 
non-sv^mmet ric  y  and  as  recursive  or  non-recursive .  A 
symmetric  non- recursive  filter  is  one  in  which  all  b(k)  eaual 
zero  and  all  a(k)  =  a(-k)y  such  as  the  filter 

x(t)  =  0,2  z(t-l)  f  0,6  z(t)   +  0.2  z(tfl). 

An  example  of  a  recursive  filter  is 

K(t)  =  a  z(t)  f  b  x(t~-l)    0<a<:iy  b  =  l-a 

which  is  not  symmetric.  This  particular  filter  may  be 
oppressed  as 

yAt)    ^  a  z(t)  f  bC  a  z(t-l)  +  bL  a  z(t-2)  +  ,,.,33 

which  reduces  to 


(t)  ^  a  z(t)  +  ab    z 


(t-1)  +  ab^z(t-2)  +.,.i  ab'^z( 
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The  recursive  filter  extends  to  the  infinite  pastr  slthouj^h 
the  coefficients  3b  will  approach  zero?  if  |bl<l»  In  this 
case?  3  recursive  filter  can  be  closel^:^  approximated  by  a 
non -recursive  filter*  A  primary  advantasie  of  the  recursive 
filter  is  that  old  data  need  not  be  stored*  New  estimates 
may  be  computed  simply  and  rapidly  as  time  evolves*  This  is 
an  important  advantage  for  real-time  applications* 

2*   Applications  of  Didital  Filters 

Di.dital  filters  are  used  to  separate  a  signal  from 
noisey  to  separate  various  freouency  components  of  a  signal  i* 
and/or  to  perform  such  mathematical  functions  as  integration 
and  differentiation*  A  review  of  Simpson's  rule  and  the 
Trapezoidal  rule  should  convince  the  reader  that  these 
numerical   intei^ration   techriic<ues  arey   in   fact?   recursive 

digital  filters*   Sometimes   a  filter  has  two   purposes*   For 

•I 

examples'  it  midht  be  desirable  v  in  estimatini^  velocity  from 
successive  observations  of  position y  to  simultaneously 
differentiate  and  remove  hisJh  freouency  noise*  When  a  filter 
is  used  to  stop  part  of  the  frequency  spec t rum y  it  is 
referred  to  as  a  "low-pa5s*y  "hij=Jh~pas5 "  y  "band-pass  •  y  or 
"band-stop"  filtery  depending  on  its  function* 

3 ♦   Analysis  of  Didital  Filters 

In  the  time  domain y  a  digital  filter  is  described 
completely  by  its  impulse  response  function y  which  is  nothing 
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more  than  the  response  of  the  filter  to  data  consisting  of  a 
string  of  crerosi'  a  sin:^le  oney  followed  by  zeros*  The  output 
of  the  filter  is  then  simply  the  wei.dhtind  coefficients  3(k)* 
If  the  filter  is  recursive  f  we  mi.dht  not  he  able  to  deduce 
the  recursive  form  from  tho?  coefficients  aik)  ■/  but  that  will 
not  concern  us  here.  The  Fourier  transform  of  the 
imp'.ilse -response  function 


N 

H(v)  =  '^3(k)    e;cp(-ivk) 


will  completely  specify  the  freauency  response  of  the  filter. 
If  the  filter  is  symmetric!'  there  will  be  no  imafrlinary  partf 
and  hence  no  phase  shift.  If  the  filter  is  recursive  i-  it 
cannot  practically  be  symmetric y  and  the  summation  will 
siene rally  run  from  zero  to  infinity.  That  isr  the  impulse 
response  will  extend  infinitely  far  into  the  future ^  which 
means  that  the  filter  remembers  all  of  the  past. 

The  duality  of  the  time  and  freauency  domains  allows  us 
to  specify  a  desired  freauency  response  and  to  design  an 
appropriate  filter  by  calculatinsi  filter  weis^hts?  or  to 
analyze  an  existing  filter  by  calculating  the  freauency 
response  from  the  filter  weidhtin<^  coefficients. 

F.   DATA  ANALYSIS  AND  EXPERIMENT  DESIGN 

No  digital  filter  should  be  applied  to  data  analysis 
without  a  clear  idea  of  the  effect  of  the  filter  upon  the 
data.    Slutsky  and   Yule   first   noted  that   some   smoothing 
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formulas  induced  periodic  functions  in  the  smoothed  estimate 
thst  were  more  the  effect  of  smooth in^^  than  of  the  original 
data  C  reft  73  ♦  A  spectral  analbisis  of  representative  raw  data 
can  be  helpful  in  deciding  on  an  appropriate  filterini^ 
techniaue*  However?  such  data  as  economic  time  series  or 
weather  data  are  typically  very  noisyy  are  based  on  a 
relatively  short  run  of  data?  and  cannot  be  described  by  an 
adequate  technoloi^ical  model*  The  analyst  must  be  aware  of 
these  problems*  Sometimes  there  are  no  siood  solutions?  but  a 
spectral  representation  may  produce  freauencies  that  can  be 
explained  on  rational  £2 rounds* 

Another  potential  pitfall  is  a  result  of  the  samplins^ 
theorem*  Consider  the  timely  e);ample  of  an  air  pollution 
model*  It  would  be  reaonable  to  suspect  that  air  pollution 
would  follow  at  least  a  daily  cycle?  or  perhaps  an  eisiht  hour 
cycle  if  mornind  and  evening  rush  hours  were  considered* 
Daily  samples  of  air  pollution  could  not  hope  to  uncover 
cycles  of  a  shorter  period  than  every  two  days*  Samples 
every  four  hours  would  be  marginally  adequate*  Hourly 
samples  would  be  necessary  for  a  ^ood  analysis*  Additionally? 
recall  the  reauirement  for  eoually-spaced  sampling  intervals* 
For  various  reasons?  the  analyst  may  h3\/e  no  control  over 
data  collection*  However?  he  must  always  understand  what  has 
been  done?  or  could  have  been  done?  to  the  data?  as  well  as 
what  he  is  doind  to  it?  in  order  to  avoid  erroneous 
conclusions  * 
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Ill*   THE  LINEAR  DISCRETE  KALMAN  FILTER 

A*   DESCRIPTION 

The  linear  discrete  Kalman  filter  is  3  recursive  Bayesisn 
least-SQuares  estimator  of  the  state  vector^  of  a  linear 
system  based  on  a  vector  of  noisy  measurements  made  at 
discrete  time  intervals*  The  process  to  be  estimated  is 
assumed  to  be  an  n -state  Gauss-Markov  process  of  order  ky 
subject  to  process  noise  U  with  zero  mean  and  covariance 
matrix  Q*  The   process  is   observed   by   an   m-dimensional 

measurement »  subject  to  measurement  noise  V  (not  to  be 
confused  with  frequency  (v))  with  zero  mean  and  covariance 
matrix  R»  The  filter  reauires  a  prior  Bayesian  estimate  of 
system  state  and  covariance*  The  recursive  estimate  of  system 
state  at  time  t  is  obtained  by  the  formula  ■ 

X(t:t)=  X(tlt"-1)  f  K<t)CZ(t)~HX(tlt-l).l 

where 

X(tlt)  state  estimate  based  on  current  measurement 

Z(t)  current  measurement 

X(t!t-1)  state  estimate  prior  to  current  measurement 

K<t)  Kalman  da in  matrix  (to  be  discussed  later) 

H  observation  matrixy  which  is  constant 

The  derivation   of  the   Kalman  filter   eauations  may   be 
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found  in  Gelb  Cref»  411  ♦  A  summary  of  the  filter  eauations  i<s 
presented  in  fi:^ure  2?  which  should  be  consulted  in  order  to 
follow  the  subseouent  discussion* 

In  General y  the  state  model  represents  a  dynamic  system? 
that  isi'  one  which  chansles  with  time*  The  extrapolation  of 
the  state  estimate  to  the  time  of  the  next  observation  is 
obtained  by  the  formula: 

X(tfl!t)  =  §    X(t!t) 

where  ^  is  the  state  transition  matrix*  The  observation 
process  occurs  accordin<2  to  the  conceptual  relation 

Z(t)  =:  HX(t)  +  V 

where  X(t)  is  the  true  system  state  observed  throu.<5h  the 
observation  matrix  Hy  and  M  represents  measurement  noise? 
which  is  assumed  to  be  a  Gaussian  random  variable  with  zero 
mean  and  covariance  matrix  R*  Note  that  the  process 
represented  by  this  formula  is  assumed  to  occur  in  the  real 
world*  The  computation  does  not  occur  in  the  filter*  Rather y 
the  measurement  Z(t)  is  an  input  to  the  filter* 

In  the  linear  Kalman  filter?  the  ^ain  K(t)  does  not 
depend  in  any  way  on  the  data*  It  depends  only  on  the  model? 
and  is  therefore  extremely  sensitive  to  assumptions*  Gain  is 
calculated  accordini3  to  the  formula 


T  T       -1 

K(t)  =■■    P(t)H  CHP(t)H   f  R3 
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where  P(t)  is  the  covsriance  in  the  system  state  estimate 
prior  to  the  current  measurement  and  R  is  the  covariance  of 
the  measurement  error*  The  covariance  is  updated  according 
to  the  formula 

T(t)    =  L"I-K(t)H3  P(t) 

where  S(t)  is  the  state  covariance  5iven  the  current 
measurement*  and  I  is  the  identity  matrix*  The  covariance  is 
extrapolated  to  the  time  immediately  prior  to  the  next 
observation  by  the  formula 

P(tfl)  =  $  2.(t)  §^i    Q 

where  ^  is  the  state  transition  matrix  and  Q  is  the 
covariance  of  the  process  noise*  Combinin.d  the  above  two 
eauations  shows  that  the  covariance  of  the  state  estimate  at 
the  time  of  the  current  measurement  depends  on  the  previous 
covariance  accordin.d  to  the  formula 

2(t)  =  i:i--K<t)H:3  C(|  Z(t-l)  i^f  Ql 


Filter  performance  is  very  dependent  on  adequate 
model  lin.'^y  particularly  on  the  state  transition  model  $  and 
the  choice  of  noise  covariances  R  and  Q*  To  a  lesser  extent? 
performance  also  depends  on  the  initial  estimates  of  system 
state  X(1!0)  and  covariance  P<1!0)*  However f  the  latter 
parameters  ar^i    less  important   because  their  effects  decrease 
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with  tiiTie.  If  the  matrices  Rv  Qy  5r  and  H  are  constant  in 
timej'  the  ^^ain  K(t)  and  covariance  matrices  ^(t)  and  P(t) 
eventually  reach  a  steady  state y  and  are  completely 
determined  by  R y  Qy  ^?  and  H* 

For  a  'fiiven  linear  filtery  it  will  be  shown  that  filter 
dainy  covariancey  and  freauency  response  will  be  completely 
determined  by  the  choice  of  R  and  Q» 

B.   THE  SCALAR  KALMAN  FILTER 

The  multi-state  Kalman  filter  is  a  powerful  computational 
device*  Howevery  it  is  difficult  and  often  impossible  to 
manipulate  in  closed  form  because  of  the  freauent  occurence 
of  singular  matrices.  An  analysis  of  a  sins^le-state  (scalar) 
filter  can  be  used  to  illustrate  the  mechanics  of  the  Kalman 
filtery  and  to  aid  in  developing  an  intuitive  understandiri:^  ♦ 
In  the  discussion  that  followsy  it  is  assumed  that  all 
matrices  are  scalarsy  andy  in  particulary  Q  and  H  eGual  one* 
Matri;-.'  notation  is  preserved  for  clarity*  Derivations  may  be 
found  in  append i;<  A» 

1  ♦    Transient  and  Steadyj-State  Gain 

It  can  be  shown  (appendix   A)  that  the  scalar  Kalman 
sJain  can  be  expressed  recursively  as 

K(t)  ^      K(t-l)  f  Q/R 


K(t-l)   -f-   Q/R  f  1 


When  the   filter  reaches  steady-state y   the  ^ain   is  constant 
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and 


2R   J 4R*   R 


The  inverse  relationship  is 


R    1-K 


Thusy  the  vari<3nce  ratio  Q/Ry  which  is  the  ratio  of  process 
noise  variance  to  measurement  noise  variance*  completely 
determines  the  steady-state  i2ain»  The  steady-state  filter  is 
completely  described  by  the  formula 

X(t)  --=  K  Z(t)  f  (1-K)  X(t-l) 

2  >   Frequency  Response 

Lettind  K   =  a   and  (1~K)   ■=  by   the  impulse- response 
function  G(t)  may  be  written 

t 
G(t)  -  ab  y  t  =  Oy 1 y2y  ♦  ♦  ♦  .  ♦ 

The    Fourier    Transform    is 


H(v) 


■J 


G(t)       exp    (-ivt)dt 


H(v)     =    3  2.  lib    exp(-iv)3 
i'o 

H<:v)    =    a    /    i:i    -    b    e>cp(-iv)3 
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Since  the  filter  is  not  symnriet  ric  y  the  freQuency  response 
H(v)  h3S  both  resl  and  iinadinary  parts*  The  amplitude  may  be 
written 


A  =   I H ( V )  I   =  yH(v)l-l(-v)  =   a  /  /  1  f  b^  -2b  cos 
which  reduces  to 


A  --=  /        Q/R 

y  Q/R  f  2(1  -cos  V) 


The  Phase  andle  may  be  written 


8(v) 


arctan  /  -b  sin  v  | 
[l    -b  cos  v/ 


The  andle  for  maximum  phase  shift  is 


v(max  9)  =  arccos  b  =  arccos  (1~K) 


V  (ma)-;  8)   =   arccos  (1  f  a/2R 


JQ^/AR^ 


■f   Q/R  ) 


Therefore  7  the  variance  ratio  Q/R  also  completely  specifies 
the  steady-state  freauency  response  of  the  filter*  Amplitude 
and  phase  relationship  for  several  values  of  ^ain  Bre  plotted 
in  figures  3   and  4* 

It  is  evident  that  hidh  Q/R  (hi.?ih  ^ain)  reduces  the  phase 
lad  of  the  filter  but  allows  more  of  the  hidh-f reauency 
components  to  pass*  Conversely y  low  Q/R  (low  dain) 
attenuates  more  of  the  hidh-f reouency  components «  at  the 
evjpense  of   an  increased  phase  lad*    Note  that  even   at  very 
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FREOUCHCY     RESFOfriE     OF     SCALAR     KAUM«M     FIUTER 
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low  ^3.in  (low  Q/R)  not  all  of  the  hi^h -f  reouency  component  is 
attenuated!'  and  the  phase  laii  is  ouite  severe*  The  slope  of 
the  amplitude  chan?^e  is  Quite  shallow?  imply  ini.^  that 
attenuation  increases  Gradually  as  freauency  increases ♦  This 
is  a  conseouence  of  the  assumptions  and  performance  will  not 
he  adeauate  if  the  data  does  not  represent  a  Gauss-Markov 
process  y  but  in  fact  represents  some  phenomena  chani^in.^  with 
time  ♦ 

C*   IMPROUIMG  THE  FILTER 

The  transition  band  of  the  filter  can  be  sharpened?  and 
more  of  the  hi^^h-  freauency  components  eliminated*  by  runnini^ 
the  data  throush  two  filters  in  series*  The  basic  scalar 
filter  was 

x(t)  ==  a  2(t)  f  b  x(t-l) 

where  a  ■-  K  and  h  =  (1-K)»  Runnind  the  data  throu£^h  the 
filter  aiiain?    we  obtain  a  new  estimate  y(t)  »  where 

y(t)  =::  a   x(t)   f  b  y(t~l) 

It  should  be  evident  that  we  can  accomplish  this  all  in  one 
step  as 


y(t)  ==  a^z(t)  f  2b  y(t-l)  -  b  y<t~2) 
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We  need  onl'-i  to  save   one  additional  previous  estimate  y(t~2) 
as  well  as  y(t-l)»   The  impulse  response  function  is 


d  <  t )  =  <  t  f  1 )  a'  b*"  y  t  ••=  O'jlf  *  *  * 


We  have  performed  a  convolution  in  the  time  domain ?  which 
corresponds  to  a  multiplication  in  the  freauenc*:^  domain. 
This  may  not  be  ejcactly  what  we  want.  Let  us  suppose  that  we 
want  the  wei^htind  coefficient  for  the  present  data  point 
s(t)  to  be  0»27  in  both  cases*  This  reauires  a  =  0.27  for 
the  basic  filter  and  a  =  ^0.27  -  0.52  for  the  double  filter. 
The  impulse  response  function  for  both  filters  is  presented 
in  fiiJure  5.  Note  that  the  double  filter  f onsets  the  past 
more  readily.  The  amplitude  and  phase  shift  for  both  filters 
is  presented  in  figures  6  and  7.  The  da in  for  the  scalar 
filter  was  0.27y  corresponding  to  a  variance  ratio  (Q/R)  of 
0.1.  Also  note  that  the  double  filter  has  somewhat  better 
hidh-f reauency  attenuations'  somewhat  less  attenuation  at  low 
freouenciesy  and  a  slightly  sharper  (steeper)  transition.  We 
would  therefore  expect  it  to  be  a  bit  better  at  separatin.d  a 
low-f reauency  signal  from  noise.  At  low  freouencies»  the 
double  filter  has  less  phase  shift.  Howevery  phase  shift  is 
more  severe  at  freauencies  above  0.5  radians. 

D.  performance:  comparison 

The  basic  scalar  filter  and   the  improved  (double)  filter 
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wero  coniF'Bred    usind   the  data   illustrated  in  figure    B,  A 

series  of  independent  normal  random  numbers  with  zero  mean 
and  variance  ten  were  Generated?  to  simulate  measurement 
noise*  They  were  added  to  a  nominal  function  which  was  a 
combination  of  steps  and  a  ramp*  The  ramp  function  rises  1 
unit  each  measurement  interval r  which  corresponds  to  one 
standard  deviation  in  process  noise  Qy  so  at  least  durinsJ  the 
ramp  functions  the  data  corresponds  to  the  filter  design 
variance  ratio  Q/R  •=  0«1*  A  spectral  analysis  of  the  data  is 
presented  in  figure  9.  The  signal  and  noise  are  presented 
separately  and  in  combination.  Note  that  the  noise-only  scale 
is  expanded*  The  noise  spectrum   is  irre-^ulary   but  overall 

Quite  flat*  The  sissnal  consists  mostly  of  very  low 
f  reauencies  y  but  also  has  some  hisih  freauencies*  This  would 
be  expected?  since  step  and  ramp  functions  reauire  very  hi<3h 
freauencies  in  their  Fourier  expansion*  The  hi:^h-f reauency 
si.dnal  is  submerged  in  noise*  The  dataf  of  course?  does  not 
fulfil  the  assumptions  from  which  the  Kalman  filter  is 
derived*  However?  the  real  world  seldom  does  either*  We  are 
looking  for  robustness* 

The  filters  were  first  tested  on  the  siilnal  alone*  The 
results  are  presented  in  fissure  10*  It  can  be  seen  that 
neither  filter  can  respond  instantaneously  to  the 
discontinuities  in  the  function?  since  hi.dh  freauencies  are 
attenuated*  Both  filters  lasl  after  discontinuities  and 
durind  the  ramp  rise*  This  is  a  conseauence  of  the  non- 
symmetric  nature  of  the  filters  and  illustrates  the  phase 
lad*    Note  that   the  double   filter  performs   a  bit   better? 
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remainini^  closer  to  the  data  throu<^hout  (it  is  a  peculiarity 
of  the  f-lottiri;5  routine  that  only  one  symbol  will  he  plotted 
when  ever  data  points  coincide*  Therefore!'  data  points  that 
do  not  af^psav  should  be  regarded  as  occurring  simultaneously 
with  the  ones  that  do  appear) * 

The  filters  were  then  tested  on  the  noisy  data*  The 
results  are  presented  in  figures  11  and  12*  The  latter  plot 
has  the  data  suppressed  so  that  the  scale  can  be  increased 
and  more  resolution  obtained* 

The  same  trends  can  be  observed  as  were  previously*  The 
double  filter  la^s  less  during  the  trend  and  transitions* 
The  double  filter  appears  to  follow  the  noise  a  bit  more 
closelyy  but  overall  it  follows  the  sisJnal  better  than  the 
basic  filter*  The  averas^e  variance  between  the  sisinal  and  the 
filter  was  4*20  for  the  double  filter  and  5*10  for  the  basic 
filter?  which  was  an  1S%  improvement  for  this  simple 
modification*  The  improvement  is  due  to  the  fact  that  the 
double  filter  weights  more  recent  data  more  heavilyy  and 
remembers  less  of  the  past  than  the  scalar  Kalman  filler? 
even  thousJh  the  weisJht  on  the  present  observation  is  the 
same  * 

This  simple  experiment  is  only  intended  to  acquaint  the 
reader  with  possible  improvements  to  the  Kalman  filter*  Like 
any  tooly  the  Kalman  filter  should  not  be  applied 
indiscriminantly ♦  The  interested  reader  is  referred  to 
Hamminss  Cref*7n  for  the  basic  principles  of  digital  filter 
desi:3n* 
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E,   THE  TRANSIENT  CASE 

The  sophisticated  reader  has  no  doubt  noticed  that  the 
steady-state  scalar  Kalman  filter  is  eauivalent  to  the 
BoK -Jenkins  IMA(Oi'li'l)  model  Cref»  IIU  and  to  Brown's 
exponential  snrioothin<3  model  Cref»10ZI*  Zehna  critici:zed  the 
exponential  smoothing  model  Zref  *131 f  notind  the  bias  would 
occur  if  the  steady-state  model  was  applied  with  an 
inappropriate  prior  estimate*  Bessler  and  Zehna  L"ref.l4II 
developed  a  ^ain  schedule  which  they  call  finite  exponential 
smoothing*   Their  formula  for  ^ain  is 


a(t)  =  a/d-b"*^  ) 


where  a  is  the  steady  state  .'^ainy  b  =  (l-a)t-  and  3(t)  is  the 
s^ain  schedule  as  a  function  of  time*  It  is  similar  to  the 
Kalman  .<3ain  schedule  if  the  initial  Kalman  ^ain  K(0)  is 
chosen  as  one.  In  both  models?  no  prior  estimate  is 
reauired*  The  weisJht  on  the  first  observation  is  one*  A 
comparison  of  the  two  models  is  illustrated  in  fi<^ure  13 y  for 
a  steady-state  siain  of  0*2  and  an  initial  slain  of  !♦  The 
Kalman  Sain  was  calculated  accordin.^  to  the  recursive  formula 
in  section  III .A* 

The  Kalman  filter  dain  converges  faster?  althou:^h  the 
difference  is  not  dreat*  The  scalar  Kalman  filter  possesses 
two  other  advantasies  over  the  finite  exponential  smoothing 
techniaue*   Firsts'  if   a  dood  prior  estimate   does  exist?  the 
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Kalman  filter  allows  the  initial  ^ain  to  be  chosen  as  less 
than  one?  and  its  value  will  be  determined  bv-i  the  assumed 
covariance  of  the  prior  estimate*  Alsoy  the  Kalman  model 
forces  the  analyst  to  at  least  think  about  the  concepts  of 
measurement  noise  and  process  noisey  and  to  estimate  the 
noise  variance  ratio  Q/R* 

As  noted  before y  a  freauency  analysis  of  the  transient 
case  is  not  appropriate*  However y  it  can  be  thought  of  as  a 
case  of  transient  bandwidth*  The  filter  is  initially  set  to  a 
hii^her  :3ain  than  steady-state  siain*  If  there  is  no  prior 
estimate  available?  the  initial  ^ain  is  oney  and  the  filter 
is  initially  an  infinite-bandwidth  or  all -pass  filter.  As 
data  are  acauired  sSain  drops  and  the  bandwidth  narrows  until 
steady-state  conditions  are  achieved*  The  concept  of 
transient  bandwidth  is  important  to  the  subject  of  adaptive 
filterins^y  to  which  we  will  return* 

F*   HIGHER  ORDER  FILTERS 

The  main  beauty  of  the  Kalman  filter  is  not  in  its 
statistically  unbiased  method  of  calculating^  <^ainy  but  in  its 
powerful  matrix  formulstiony  which  allows  it  to  be  applied  as 
a  multi-dimensional  model  incorporating  any  order  of 
differencing  desired*  As  the  state  space  is  increasedy  it 
auickly  becomes  impossible  to  analyse  the  filter 
analytically*  Hi<3h-order  multi -dimension  filters  can  also 
easily  exceed  the  capacity  of  present  digital  computers  for 
real-time  applications*   Fortunately  y  it   has  been  found  tihat 
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the  state  spscg  can  be  reduced  and  dimensions  decoupled  with 
very  little  degradation  in  the  overall  accuracy  of  the  state 
estimate  llrefs*  4  and  51*  For  e>;ample!'  if  a  12--state  model 
can  be  reduced  to  9  states  and  can  be  adeauately  represented 
bv^  three  3-state  models »  the  matri;c  calculations  can  be 
considerably  simplified  and  speeded  up* 

We  will  examine  a  second-order  (first  difference) 
filter?  which  can  be  used  to  estimate  trend?  or  velocity.  We 
will  use  the  latter  term*  Position  and  velocity  are  to  he 
estimated  based  only  on  successive  measurements  of  position. 
The  state  transition  and  the  observation  matrices  are 


§. 


[;;] 


H  =  CI  0.1 

The  covariance  matrices  i?  P?  and  Q  are?  of  course?  2  by 
2  matrices.  The  state  vector  has  two  elements?  velocity  and 
position?  while  the  measurement  vector  has  only  position. 
The  measurement  error  R  is  a  1  by  1  matrix  which  we  will 
vary.   We  have  chosen  Q  as 


Q 
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arcluiri^  that  any  process  noise  will  be  contained  entirely  in 
velocity.  That  is?  there  can  be  no  random  motion  that  is  not 
caused  by  a  random  velocity.  Randomness  of  velocity  will  feed 
into  position  throus^h  the  state  transition  matrix. 
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Even  in  this  simple  C3se»  solving  an3l\-jticall\:f  for 
stesd'd-state  ^3 in  in  teriTis  of  R  and  Q  reauires  solving  a 
system  of  4th  order  polynomial  equations.  We  will  opt 
instead  for  a  computer  solution*  The  reader  may  continue  to 
think  in  terms  of  the  noise  variance  ration  where  R  will  take 
on  the  values  ly  10?  100  and  Q  will  remain  constant  as  above* 
Since  there  is  only  one  non-zero  term  in  the  Q  matrix y  we  may 
think  of  the  noise  variance  ratio  as  the  scalar  Quantity 
a<2?2)/R.   The  resulting  steady-state  ^ains  are 
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As  would  he  expected^  the  position  ^ain  is  much  higher 
than  that  of  a  scalar  filter  at  an  eauivalent  noise  variance 
ratio?  because  the  process  variation  now  applies  to  velocity 
rather  than  position*  The  velocity  da in  is  considerably  less 
than  the  position  ^ain?  since  the  velocitvi  is  not  measured 
directly  but  must  be  estimated  from  successive  measurements 
of  position ♦  The  impulse-response  function  of  the  medium-i^ain 
filter  (noise  variance  ratio  0*1)  is  presented  in  fisiure  14* 

The  amplitude  response  of  these  three  filters  is 
compared  in  fii^ures  15  and  16*  The  most  striking  feature  is 
the  amplification  which  occurs  at  a  specific  freauency  in  the 
position  freauency  response*   This  implies  that  the  filter  is 
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FRfCGUei/C  r      RtlSFONSli     OF      POSTTION     ESTIMATE 
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most  sensitive  to  (notion  in  a  particular  freauency  r3n.<^e» 
Thusy  the  natural  freauency  of  the  system  to  be  observed c  if 
it  is  known  y  is  a  sisSnificant  desi^^n  parameter*  Adainx  we 
observe  the  attenuation  of  hidh-f reauency  noises*  althou<2h  a 
sii^nificant  amount  still  remains  at  the  maximum  freouency 
(note  that  fiirlure  16  does  not  include  the  ori.'^in)* 

The  freauency  response  of  velocity  shows  a  reduction  in 
amplitude  at  low  freouency*  The  amplitude  at  zero  freauency 
is  eaual  to  the  velocity  <3ain»  This  is  far  from  ideal 
performance  for  a  differentiator  7  which  should  have  an 
amplitude  response  of  zero  at  zero  freauencyy  with  a  slope  of 
one  UP  to   the  cutoff  freauency  Zref  *71 ,  The  differentiator 

isy  however y  reasonably  effective  at  reducing  the  amplitude 
of  hidh-f reauency  components* 

The  Phase  shift  of  the  filters  adain  shows  increasing 
phase  laid  as  dain  is  decreased*  The  overall  effect  is 
similar  to  the  scalar  filteri'  and  is  otherwise  unremarkable* 
Thereforey  plots  are    not  included* 

The  data  of  fii^ure  8  was  tested  on  the  lowest-i^ain 
velocity  filter*  Note  that  the  spectral  content  of  the  data 
(fisJure  9)  is  Guite  low-f reauency y  and  that  the  bandwidth  of 
the  lowest  i^ain  velocity  filter  is  auite  widey  and  indeed  is 
higher  than  that  of  our  scalar  filter*  So  it  midht  be 
expected  that  the  velocity  filter  would  have  some  trouble 
with  the  data* 

The  velocity  filter  performance  on  the  nominal  function 
only  is  presented  in  figure  17*  The  filter  overshoots  auite 
badly  at   the  discontinuous  steps?   which y  of  course y   are  an 
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inripulse  in  velocity*  The  overshoot  is  less  at  the  start  and 
stop  of  the  trend.  It  does  settle  down  and  track  the  trend 
without  Issli'  which  is  an  inriprovement  in  perf orinance  over  the 
scalar  filter*  It  should  be  noted  that  with  hisJher  dainy  the 
filter  would  track  the  nominal  data  better y  while  with  lower 
Gains'  the  overshoots  would  be  more  severe* 

The  performance  of  the  velocity  filter  on  the  data  is 
illustrated  in  fi.'^ure  IS*  As  expected?  the  filter  tends  to 
follow  the  noise  too  much*  Howevery  it  does  follow  the 
discontinuities  much  more  ouickly  than  the  scalar  filter* 
This  points  out  the  fact  that  the  hi::3her-order  filter  is  more 
effective  as  a  maneuver  detector  but  it  is  less  suitable  for 
5moothin<3  very  noisy  data*  This  asJain  illustrates  the 
concept  of  bandwidth?  which  is  Guite  hii^h  even  in  the 
low-dain  velocity  filter* 

In  retrospect?  the  decision  to  choose  Q(lrl)  as  zero  may 
not  have  been  wise*  Allowind  some  process  noise  in  position? 
exclusive  of  velocity?  could  well  have  some  smoothing  effect 
on  the  velocity  estimate?  which  would  result  in  smoother 
one-period  ahead  predictions*  This  could  smooth  the 
operation  of  the  filter  a  bit*  The  possible  combinations  of 
filter  parameters?  even  for  this  simple  filter?  are  auite 
numerous* 

The  freouency  response  of  a  second-difference 
(acceleration)  filter  was  also  determined  for  comparison* 
The  results  are  presented  in  figure  19*  The  Q  matrix  was 
zero  except  for  Q(3?3)?  which  was  one*  R  was  chosen  as  10? 
resulting  in  a   nominal  noise  variance  ratio   of  0*1*   Ada  in? 
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the  da  in  was  hi  :3her  than  that  of  a  velocity  filter  with  an 
eauivalent  noise  variance  ratio*  The  amplification  of 
low -f  reauency  components  of  position  was  increaseds-  and  the 
zero-f reouency  amplitudes  of  velocity  and  acceleration  aiSain 
corresponded  to  filter  :3a in* 
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FRECJUEI/CY     RESPONSE     OF     ACCELERATION     FIUfER 
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IU«   ESTIMATION y  SMOOTHING y  AND  PREDICTION 


A»   ESTIMATION 

So  far*  we  have  been  concerned  only  with  estimation  of 
the  present  state*  A  filter  designed  to  provide  such  an 
estimate  cannot  be  symmetric  y  because  it  can  put  no  wei:^ht  on 
future  observations*  Thusr  phase  lasJ  is  inevitable  y  and  is 
one  of  the  parameters  that  should  be  considered  in  the  design 
process* 

B»   SMOOTHING 

Smoothing  is  the  use  of  a  filter  to  provide  an  estimate 
of  past  states*  Such  a  filter  can  be  made  symmetric^  which 
completely  eliminates  the  phase  l3<3»  Non-recursive  smoothin:^ 
filters  cause  a  loss  of  N  data  points  at  each  end  of  the 
data?  where  the  span  of  the  filter  is  <-Ny  N)» 

The  Kalman  filter  can  be  used  as  a  smoother  by  simply 
runninsJ  the  forward  estimate  through  the  filter  in  the 
opposite   direction*   The   impulse  response   function  of   the 

scalar  filter  was 

3<t)  =  ab"^ 
It   can   be  shown   (appendix^   A)   that  the   impulse   response 
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function   of   the   smoother   (forward   and   backward   filters 
combined)  is 

t 

G(t)  =  ab  /(1+b) 

* 

which  is  Just  the  convolution 

3(t)  (D  £i(-t) 

The  Kalman  filter  is  able  to  provide  an  estimate 
throu<3hout  the  span  of  the  data*  No  data  is  lost  at  either 
end*  However?  due  to  transient  effects y  the  data  near  either 
end  is  subject  to  phase  shift  and  some  increase  in  <3ain*  The 
filter  is  necessarily  not  symmetric  near  each  end  of  the  data 
span* 

Gelb  L"ref*4II  includes  a  complete  discussion  of 
fixed-point »  f  i;ced-la<2f  and  fixed-interval  smoothin53*  We 
will  restrict  our  attention  to  the  scalar?  fixed-interval y 
steady -state  casey  i.^norins5  the  end  effects* 

The  scalar  Kalman  filter  of  section  III*D  (noise 
variance  ratio  of  O.ly  <3ain  of  0*27)  was  used  as  a  smoother 
on  the  data  of  figure  8*  The  results  are  presented  in 
figure  20*  As  compared  to  the  one-pass  performance  as 
illustrated  in  fi<3ure  12y  the  smoothed  data  shows  phase  la^ 
removed  and  peaks  in  the  oscillations  reduced*  However?  the 
smoother  has  less  ability  to  follow  the  discontinuities  in 
the  nominal  function*  Tiie  removal  of  the  phase  la<2  is 
characteristic  of  any  symmetric  filter*   However?  the  reduced 
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ability  to  follow  the  discontinuities  is  the  result  of 
reduced  effective  :^sin*  Since  we  have  convolved  the  filter 
weidhtsy  we  have  sQuared  the  amr- litude  of  the  freauency 
response.  The  wei^iht  on  the  data  point  at  (t--0)  is  reduced* 
The  effective  dain  was  0,27  for  the  forward  filter^  and  0.156 
for  the  smooth in52  filter.  This  reduction  in  effective  ^ain 
is  not  addressed  in  the  literature  on  the  Kalman  filter^  and 
it  is  unclear  how  smoother  ^ain  should  he  chosen  in  relation 
to  the  noise  variance  ratio  Q/R. 

C.   A  COMPARISON  OF  TWO  SMOOTHERS 

The  Kalman  smoother  of  the  previous  section  was  comparGd 
with  a  Gaussian  smoother  to  illustrate  some  design  options 
and  procedures.  The  Gaussian  smoother  was  chosen  from  amon^^ 
a  hu53e  variety  of  data  windows  because  it  has  s2ood  smoothin<3 
properties!'  and  because  it  is  particularly  easy  to  desii^n. 
Interestins^ly y  preliminary  experiments  showed  repetitive 
applications  of  a  Kalman  filter  to  result  in  an  approximately 
Gaussian  filter  weight  distribution.  A  comparison  of  the 
Gaussian  smoother  to  a  variety  of  other  windows  is  contained 
in  Harris  Cref.1511. 

The  Gaussian  smoother  is  a  symmetric  filter  with  the 
weights  chosen  accordinsf  to  a  discretized  and  truncated 
normal  distribution.   The  formula  is 


^(t)  =  K  exp("-t^/2<r*) 
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where    t    is      an    inte53e?r    on    the       rani^e    (-Ny    N)    and      K    is    chosen 
such    that 


.d(t) 


The  ease  of  desi<^n  comes  froin  tihe  observation  that  the 
Fourier  transfornri  of  the  continuous  Normal  distribution  is 
also  a  Normal  distribution  with  scale  parameter  (variance) 
eaual  to  l/(y.  As  lon.'Si  as  (flr-  >  2)  and  truncation  is  not  more 
severe  than  Jn|  2ffv  a  reasonable  approximation  of  the 
freauency  response  for  the  Gaussian  digital  filter  is 


G(^)  Si    e>cp(  -0'*v^/2) 


The  scale  parameter  was  chosen  such  that  the  frequency 
response  was  eaual  at  (v  -  0*5)  ♦  Skippins^  the  algebraic 
details?  this  re«uired  (fi*  =  3 » 1 .1 )  ♦  The  Gaussian  smoother  was 
truncated  to  (N  ~  7)f  resultinsJ  in  a  filter  span  of  15  data 
points.  The  freGuency  response  of  both  filters  is  presented 
in  figure  21!-  and  the  filter  weisJhtinsJ  coefficients  are 
presented  in  fidure  22. 

Since  we  truncated  the  Gaussian  smoother f  we  would 
expect  some  ripples  in  the  tail  of  the  freauency  response r 
which  are  Just  barely  visible  in  fidure  21.  The  Gaussian 
filter  has  a  sharper  transition  band?  and  is  wuite  effective 
in  blocking  hisJh  freauencies*  As  compared  to  the  Kalman 
smoothers  the  Gaussian  filter  wei.<^ht5   the  present  data  point 
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less/  nearby  dabs  points  more  and  farther  data  points  less* 

The  performance  of  the  Gaussian  smoother  on  the  data  of 
figure  3  is  presented  in  figure  23*  A  comparison  of  the 
Kalman  and  Gaussian  smoothers  is  presented  in  fixtures  24  and 
25  ♦  In  fissure  24?  the  smoothers  are  applied  only  to  the 
nominal  function ♦  It  can  be  seen  that  the  Gaussian  smoother 
followed  the  discontinuities  and  corners  of  the  nominal 
function  better  than  the  Kalman  smoother ♦  However y  when  the 
smoothers  were  applied  to  the  noisy  data?  the  results  were 
less  clear  (figure  25).  The  Gaussian  smoother  as^ain  followed 
the   nominal  function   a   bit  betters"   but   it  also   followed 

low-f reauency  components  of  the  noise  a  bit  more?  tending  to 
emphasize  cyclic  effects  that  aren't  really  there.  The  mean 
SQuare  difference  between  the  smoothed  estimate  and  the 
nominal  function  were  very  similar)-  1.81  for  the  Gaussian 
smoother  and  1.34  for  the  Kalman  smoother.  Thusf  the  Kalman 
filter  seems  auite  effective  when  used  as  a  smoother.  The 
reader  is  reminded  that  the  mean  s«uare  difference  for  Vne 
scalar  Kalman  filter  was  5.1y  which  clearly  indicates  the 
superiority  of  smooth  ins'  over  filtering. 

D*   PREDICTION 

Prediction  is  difficult.  Recall  that  a  stochastic 
process  is  an  ensemble  of  possible  pathsy  while  data  is  the 
manifestation  of  one  member  of  that  ensemble.  What  could 
have  happened  did?  but  what  can  happen  isn't  necessarily 
.^oind  to.   Prediction  can  be   thoujSht  of  as  filtering  without 
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messureiTiGnts  y  and  the  estimste  is  projected  forward  in  time 
throusJh  the  state  trsrisition  model*  Filtered  or  smoothed 
estimates  may  be  auite  accurate!'  even  if  the  state  transition 
model  is  not.  However?  dood  predictions  ai'e  heavily 
dependent  on  an  accurate  state  transition  model* 

The  hi?iJher-order  Kalman  filters  are  polynomial  models* 
Hammings  has  pointed  out  that  polynomial  models  are  poor 
predictors y  since  the  estimate  tends  to  veer  off  to  plus  or 
minus  infinity  as  soon  as  the  model  is  released  from  the  data 
Cref«7;i,  There  is  no  reouirement  to  use  the  same  model  for 
prediction  as  for  filterinsj.  For  example?  it  mi<3ht  make  sense 
to  track  a  tar.<3et  with  an  acceleration  filter?  but  to  compute 
fire  control  information  based  on  a  constant-velocity  model? 
since  target  acceleration  is  £3en(3rally  assumed  random  with 
nero  mean.  Similarly!/  the  economist  may  desire  to  filter 
data  with  a  hidh-order  models  but  make  predictions  based  on 
constant  trend,  Clark  Crcf.5II  discusses  a  somewhat  more 
sophisticated  method  due  to  Sin?ier?  in  which  the  model  decays 
from  an  acceleration  predictor  to  a  constant-velocity 
predictor  as  prediction  time  increases.  Such  techniaues  are 
heuristic  in  nature?  but  can  prove  valuable?  to  the  innovative 
analyst . 

An  interesting^  example  of  the  above  concept  can  be  found 
in  Box  and  Jenkins  Cref.llII.  They  compared  a  quadratic 
forecast  due  to  Brown  Cref.103  with  their  own  IMA<0?1?1) 
model  with  a  sJain  of  0.9.  The  latter  model  is  eauivalent  to 
the  steady-state  scalar  Kalman  filter.  The  data  used  for  the 
comparison  was   a  time   series  of  IBM   stock  prices.   Box  and 
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Jenkins  observed  thaty  while  the  Quadratic  model  mi^ht  well 
be  used  to  fit  the  data?  its  performance  as  a  predictor  was 
clearly  inferior  to  the  simpler  IMA(Oyl!»l)  model*  This  is 
not  surprisin<=l  •  since  it  has  lond  been  su:3i3ested  that  stock 
prices  behave  as  a  randorfi  walk?  and  that  the  best  forecast  of 

stock  pricey  at  least  in  the  short  runy  is  the  present  price 
Cref»llll»  Note  that  the  foredoin'2  implies  that  the  sJain 
should  be  set  to  l»Oy  which  corresponds  to  no  filtering  at 
all*  Therefore  y  E<ox  and  Jenkins  apparently  found  that  some 
filterinsS  of  the  data  was  sr-'pvop'viate  f  even  thousJh  the  :^ain 
they  used  was  auite  hii^h* 
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V.  some:  refinements y  extensions  and  alternatives 

A,   ADAPTH^E  FILTERING 

In  the  linear  Kslman  filter?  the  slain  is  completely 
independent  of  the  data*  Clearly?  this  will  result  in  maJor 
errors  if  the  i^ain  is  chosen  inappropriately  or  if  the  data 
statistics  change*  If  the  :3ain  is  too  low?  the  filter  la.<3s 
badly*  In  the  extreme  case?  which  occurs  if  the  process 
noise  covariance  Q  is  much  too  low?  the  filter  pays  much  too 
much  attention  to  the  past  and  diverges  from  the  data*  On 
the  other  hand?  if  the  iHain  is  too  hi3h?  the  filter  pays  too 
much  attention  to  the  data  and  the  state  estimate  contains 
noise.  If  the  filter  is  a  polynomial  model  and  is  to  be  used 
as  a  predictor?  the  resultins^  errors  will  be  spectacular* 

The  solution  is  simple  in  concept  but  can  be  difficult 
to  implement.  One  simply  sets  the  steady -state  ^ain  as  low 
as  appropriate  for  the  stable  process  bein^  estimated.  In 
target  trackings?  the  s^ain  would  be  set  to  track  an  airplane 
flying  a  straisiht  path.  A  "maneuver  detector"  or  "trend 
detector"  is  incorporated?  which  is  nothini^  more  than  a 
recursive  statistical  test  applied  to  the  residuals  to 
determine  whether  or  not  they  come  from  a  zero-mean 
distribution.  If  not?  the  bandwidth  is  iSradually  widened 
(^ain  is  increased)  until  the  residuals  pass  the  zero-mean 
test.  Then?  the  da in  is  allowed  to  decrease  toward  the 
stable?  steady-state  value. 
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Further  detailis  and  some  novel  <3f='p roaches  are  discussed 
in  Clsrk  C  reft  53  ♦  Two  exsinples  taken  from  Clark  are 
illustrated  in  fidure  26*  The  conceptual  adaptive  filter 
discussed  above  reauires  time  to  detect  the  maneuver  or 
trendy  adapt  to  ity  and  reconver?3e  to  a  stable  ^ain  settind* 
Durin'ii  this  time?  the  state  estimate  is  less  accurate  y  and 
the  time  reauired  ma^:^  be  unacceptable  lonsi  for  some 
appl i cat ions ♦ 

Clark  proposes  a  dual  -bandwidth  adaptive  filter  to  speed 
adaptation*  The  process  is  simultaneously  tracked  by  a 
narrow-band  and  a  wide-band  filter*  If  a  maneuver  or  trend 
is  detectedy  the  state  estimate  of  the  wide-band  filter  is 
fed  into  the  narrow-band  filter.  Ideality  this  would  allow 
the  narrow-band  filter  to  Jump  immediately  to  the  current 
(unbiased)  estimate  of  the  wide-band  filter*  In  practicer 
Clark  found  that  some  wideninj^  of  the  bandwidth  of  the 
narrow-band  filter  was  also  reojuired* 

Voluminous  literature  exists  on  the  subject?  much  of  it 
very  difficult  to  read*  Clark  Cref*5Il  incorporates  a 
particularly  lucid  account  of  stability  problems  encounteredy 
methods  of  reducin<.;i  the  cost  of  false  detection  of  biasr 
analytical  methods  of  determining  filter  parameters y  and 
experimental  results*  Although  Clark's  filter  was  desi.^ned 
to  track  and  predict  the  position  of  airborne  tariSetsy  the 
methods  discussed  are  adaptable  to  the  filtering  of  economic 
time  series  or  virtually  any  other  stochastic  process* 
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B.   NON-LINEAR  FILTERING 

In  the  non-l inesr  Kslman  filters  one  or  more  of  the 
nri<3t rices  Q?  Ry  Hy  or  §  are  allowed  to  vary  with  time*  Since 
this  results  in  a  time-variation  of  the  ^ain  rriatrixy 
Quantitative  analysis  in  the  frcauency  domain  is  no  longer 
aF-propriate  ♦  However:'  it  is  well  to  keep  the  concepts  in  mind 
in  order  to  sJain  additional  insis^ht*  There  are  two  basic 
types  of  non-linearities  that  may  arise?  non- linear 
measurements  and  non-linear  dynamics* 

1  ♦   Non-Linear  iieasurements 

Non-linear  measurements   arise  when  observations  are 

made  in  one  coordinate  system  and  the  model  reauires  that  the 

states   be  estimated   in  another   coordinate  system*  In   this 

case/  the  matrices  R  and  H   are  time~varyin:S  functions  of  the 

I 
coordinate  t ransf ormationr  and  do  depend   on  the  datar  in  the 

sensis  that  they  depend  on  the  location  of  the  data  within  the 

coordinate  system*   This  type  of   non-linearity  is  often  easy 

to  handle* 

For  the  best  example  of   non- linear  measurements  we  must 

return  to   the  tardet-t racking   model*   Fire   control  systems 

<3enerally  track   in  azimuth"  elevation y  and   ran:^e  *   However r 

the  model  is   a  polynomial  in  Cartesian   coordinates r  but  not 

in  polar  coordinates*    Airplanes  often  fly  a   straight  pathr 

but  seldom!-   if  ever?   fly  a  constant   bearing  or   rans^e  with 

respect   to    the   radar   observer*-    In   this    model*   the 
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non-linearity  can  be  reduced  by  cansiderini3  the  Cartesian 
iTieasurement  error  as  a  linear  transformation  of  the  polar 
measurement  error*  If  the  polar  measurement  error  is 
Gaussian)'  the  Cartesian  measurement  error  is  very  nearly 
Gaussian  with  covariance  matrix  R  a  function  of  the 
coordinate  transformation* 

The  Cartesian  R  matrix  will  not  be  diagonal ^  even  if  the 
polar  R  matrix  is*  However y  Clark  Cref.SU  has  found  that 
setting  the  of f-diadonal  terms  of  the  Cartesian  R  matrix  to 
zero  did  not  appreciably  degrade  filter  performance.  In  this 
wayy  he  was  able  to  decouple  a  nine-state  filter  into  three 
three-state  filters* 

If  the  measurement  non-linearity  is  too  severe y  it  may 
not  be  reasonable  to  assume  that  the  noise  is  Gaussian. 
However?  limited  experiments  performed  on  data  with 
non-Gaussian  noise  (an  exponential  distribution  was  used) 
showed  that  the  Kalman  smoother  and  the  Gaussian  smoother 
were  Quite  robust  as  lon^  as  the  <2airi  was  not  hisJh*  This 
seems  to  be  a  conseauence  of  the  Central  Limit  Theorem?  since 
low  s$ain  implies  a  linear  combination  of  a  fairly  larde 
number  of  data  points*  It  should  be  noted  that  a  filter 
designed  to  handle  this  situation  is  still  linear?  althourJh 
the  Gaussian  assumption  is  violated* 

2 .   Non-Linear  Dynamics 

Non-linear  dynamics  are  considerably  harder  to  handle 
than  non-linear  measurements*  This  is  unfortunate?  since  the 
areas   of   potential  application  are      numerous*    Non-linear 
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dynamics   occur   when  the   Q   or   ^   matrices  depend   on   the 
previous  history  of  the  process. 

As  3   simple  example?   consider  the   multiple  resJression 
model 


y(t)  --=  3  >£,(t)  +  b  x^(t)  +  y(t-l) 


where  y(t-l)  corresponds  to  the  intercept  term.  In  this 
model)'  we  wish  to  estimate  the  dependent  variable  y(t).  To  do 
soy  we  need  to  estimate  not  only  the  independent  variables 
x,(t)  and  x(t)»  but  also  the  reilression  coefficients  a  and  b» 
Let  us  assume  we  can  measure  y<L)y  x.(t)y  and  xlt)  f  but  not  a 
and  b*  Assumin<^  a  first-order  system?  the  state  update 
eauation  is 


y(tfl It) 


x/t+l  It) 


:(j(tfl  It) 
a(t+l It) 
b<t+l It) 


1  a(tlt)  b(tlt)   0   0 

0      1  0     0   0 

0      0  10   0 

0      0  0     10 

0      0  0     0   1 


y(t I t) 


x,(  1 1 1 ) 


;  ;j(  1 1  t ) 
a  ( 1 1 1 ) 
b  ( 1 1 1 ) 


Where  the  transition  matrix  is  unfortunately  not  uniaue.  The 
first  row  of  the  transition  matrix  could  be  equally  well 
represented  by 


i:  1   0   0   x/tit)   ;i^(tlt)3 
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or  even 


C  1   3(t!t)/2   b<tlt)/2   K/t:t)/2   x^(t!t)/2  1 


and    it  obviously   changes  at  every  iteration^   It   is  at  this 
point  that  filter  desisin  becomes  an  art* 

Note  that  the  independent  variables  and  the  reiiiression 
coefficients  are  assumed  here  to  be  first-order  Gauss-Markov 
processes*   Increasingly  hi^h  orders  would  multiply  the  state 

SP3C© ♦ 

Several  experiments  were  run  usini^  a  second-order  model 
similar  to  the  above  on  the  Box -Jenkins  Cref»llll  series  M 
data  (sales  data  with  leadind  indicator)*  Quantitative 
results  are  not  presented?  because  the  Box-Jenkins  data  did 
not  include  sufficient  forecast  estimates  for  comparison f 
some  " cheat ini3"  was  done  because  the  Box- Jenkins  parameters 
were  used  in  filter  desij3ny  and  it  never  became  clear  exactly 
what  parameters  were  appropriate  for  the  R  and  Q  matrices* 
However!'  some  Qualitative  comments  are  appropriate*  The 
model  did  work*  Some  instability  was  noted  in  the  regression 
parameters*  It  became  obvious  that  tfie  sJain  on  the 
regression  parameters  must  be  set  very  low  in  comparison  to 
the  £{ain  on  the  independent  vari ablest  in  order  to  keep  the 
regression  parameter  estimates  from  varying  faster  than  the 
estimates  of  the  independent  variables*  This  implies 
choosin<3  small  values  for  the  noise  variance  of  the 
regression  parameters*   Also!-  by  keeping   the  ^ain  fairly  low 
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on  the  lesdiriiS  indicator!'  it  W3S  possible  to  induce  a  phsse 
l3d  that  3PproxiiTiste.ly  cancelled  the  lead. 

The  reaui rement  to  keep  ^ain  low  in  order  to  improve 
stability  is  evidently  a  conseauence  of  the  increased  de<3rees 
of  freedom.  The  more  parameters  to  be  estimatodc  the  more 
dei^rees  of  freedom  in  the  model*  HisJh  ^ain  is  anala^ous  to 
relatively  few  data  points  beinsJ  used  in  redression*  The 
more  variables  we  introduce  into  the  model »  the  less  ^ain  we 
ave    able  to  use* 

It  is  indeed  unfortunate  that  multiple  re<3ression  is  a 
non-linear  problem  when  cast  in  a  filter  model*  It  would  be 
useful  to  have  a  multiple  regression  model  for  which  more 
recent  observations  were  wei«3hted  more  heavily  than  older 
ones  in  determining  the  regression  parameters*  No  doubt  the 
innovative  analyst  could  develop  one  to  fit  the  specific 
situation*  However r  clearly-defined  techniques  with 
demonstrated  results  ar<s  not  available  to  the  practitioner* 
The  experts  all  have  their  favorite  methods f  and  much  of  the 
literature  is  difficult  to  read*  There  is  clearly  a  need  for 
additional  research  in  this  area* 

C*   NON-PARAMETRIC  FILTERING 

We  close  our  discussion  with  an  interesting^  alternative 
to  conventional  digital  filterin;;^  techniaues*  There  ax^e 
those  who  are  bothered  by  the  usual  distributional 
assumptions  made  in  any  application  of  parametric  statistics* 
An   extensive   literature   has   developed   in   the   field   of 
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non-p3r<3iTie trie  statistics?  which  is  based  on  the  principle 
that  distributional  assumptions  are  avoidedy  or  at  least 
weakened*  A  strond  point  of  non-parametric  statistics  is 
relative  insensi tivi ty  to  extreme  outliers*  However y  little 
pro.'^ress  has  been  made  in  the  non-parametric  analysis  of  time 

series*  An  exception  may  be  found  in  Tukey  Cref*16Ilf  which 
is  presented  in  a  hi^Shly  intuitive  manner?  with  little  or  no 
theoretical  b3ck:3round. 

One  simple  idea  advanced  by  Tukey  is  that  of  median 
smoothini^*  The  smoothed  estimate  is  based  on  the  median  of 
several  adjacent  data  points y  rather  than  on  a  weighted 
linear  combination*  The  result  is  obviously  a  series  of 
steps y  since  adjacent  data  points  will  often  have  the  same 
median.  Tukey  su^<3ests  several  methods  to  restore  some 
curvature  in  the  estimate*  These  will  not  be  developed  here* 
Tukey 's  methods  would  be  relatively  hard  to  mechanize  on  a 
computer y  because  the  methodolosJy  reauires  extensive  logical 
r  ij  1  e  s  * 

Tukey 's  methods  could  be  extended  to  real-time  filtering 
problems  by  developing  a  non-parametric  analoi^  to  the 
recursive  di:^it3l  filter*  Recall  that  the  recursive  dii^ital 
filter  consists  of  a  weighted  linear  combination  of  recent 
data  points  added  to  a  weighted  linear  combination  of  recent 
estimates*  The  non-parametric  filter  estimate  would  simply 
be  the  median  of  several  recent  data  points  and  several 
recent  states*  The  idea  is  intuitively  appeal in^S  and  should 
be  the  subject  of  future  research*  Discussion  here  will  be 
limited  to   some  of   the  more   obvious  traps   that  await   the 
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unwsry. 

If  such  3  filter  were  to  be  designed?  the 
iiripulse- response  function  would  be  mean  ins^l  ess  f  because  the 
median  estimate  would  always  be  2:ero  if  the  span  of  the 
filter  were  Greater  than  two.  Nevertheless f  a  median  filter 
does  have  a  freauency  response^  which  in  fact  is  a 
particularly  nasty  one* 

Consider  a  seven-point  median  smoother y  where  the  state 
estimate  at  time  t  is  the  median  of  the  measurements  made  at 
time  (t-3)  to  time  (t+3)»  This  is  anala^ous  to  the 
rectansJular  (parametric)  window  discussed  in  Hammins^  Cref»7II» 
The  rectangular  window  wei;3hts  all  data  points  within  the 
window  eQually.  The  median  window  obviously  does  the  same* 
As  a  result?  we  would  expect  the  freauency  response  of  the 
median  window  to  have  severe  ripples  as  does  that  of  the 
rectani^ular  window*  We  can  see  intuitively  that  this  is  true* 
Since  the  span  of  our  example  median  window  is  7f  the 
freauency  response  of  any  freauencvj  that  is  a  non-;rero 
integer  multiple  of  1/7  is  obviously  zero*  The  freauency 
response  at  zero  freauency  is  one»  since  the  zero  freauency 
implies  a  constant*  The  amplitude  of  the  freauency  response 
falls  off  to  the  first  zero»  then  rises  asiain*  Successive 
maxima  decrease  with  increasinsJ  freauency^  but  the  freauency 
response  is  always  non-zero  except  at  freauencies  that  ax^e 
non-zero  inte^^er  multiples  of  the  reciprocal  of  the  span  of 
the  window*  Thus»  the  non-parametric  filter  will  need  to 
incorporate  some  (unspecified)  device  to  improve  the 
freauency  response* 
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Two  other  difficulties  are  worthy  of  mention,  Firstr 
the  S3iriplin£3  distribution  of  the  median  may  have  a  1  artier 
variance  than  the  samplini^  distribution  of  the  mean*  This 
means  that  the  parametric  filter  may  provide  a  better 
estimate  than  the  non-parametric  filter  if  the  assumptions  on 
which  the  parametric  filter  is  based  are  at  all  reasonable* 
Second r  for  the  non-parametric  filter  to  be  useful?  the 
median  must  be  a  statistic  of  interest*  If  it  is  assumed 
that  the  distribution  is  symmetric?  the  median  and  mean  are? 
of  course?  eaual*  If  the  samplinsl  distribution  is  skewed?  the 
mean  cannot  be  deduced  from  the  median  unless  strict 
parametric  assumptions  are  imposed?  which  of  course?  override 
the  Justification  for  the  non-parametric  filter  in  the  first 
place*  The  idea  is  nevertheless  intrii^uin^?  and  should  be 
explored  further* 
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UI,   SOME  APPLICATIONS 


The  KsliTisn  filter  has  been  applied  to  Operations  Research 
and  economic  problems  with  varyini^  de^^rees  of  success* 
McWhorter  Cref.1711  conducted  an  empirical  study  of  the  Kalmsn 
filter  in  which  he  compared  it  to  several  other  methods  of 
time  series  f  orecastin<3  ♦  The  results  were  mijced?  with  no 
method  dominating  ♦  The  Kalman  filter  compared  more  favorably 
over  a  short   term  f o recast in."^  horizon  than  over   a  lonsJ  term 

one*  Its  performance  was?  not  surprisin.<^ly !»  found  to  be 
degraded  if  the  structural  model  was  seriously  mis-specif ied» 
McWhorter  pointed  out  some  of  the  difficulties  encountered  in 
building  the  model*  In  an  economic  contextr  it  is  often  very 
difficult  to  specify  the  noise  covariance  matrices  R  and  Q  i* 
and  even  to  identify  the  structure  of  the  state  transition 
matrix  $♦  The  assumptions  made  are  often  3weepin<:J  and 
arbitrary!*  in  contrast  to  trackinx^  applications  where  the 
noise  processes  and  especially  the  state  transition  model  are 
relatively  well  understood* 

A*   INVENTORY  MANAGEMENT 

The  Kalman  filter  is  directly  applicable  to  inventory 
management y  and  if  properly  designed?  should  be  superior  to 
the  finite  exponential  smoothing  model  of  Bessler  and  Zehna 
Cref*143*   Downini^y   Pike?  and   Morrison  Cref*lSIl   designed  a 
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KaliTisn  filter  for  the  inventors  control  of  nuclear  material* 
The  paper  is  readable y  and  the  filter  is  we  1 1 -documented  and 
ess'j  to  understand*  They  use  the  concept  of  a  control 
vector!'  which  has  not  been  mentioned  here.  An  interestinid 
peculiarity  of  the  model  is  that  one  of  the  measurements  is 
only  available  once  every  twenty  iterations.  The  state 
transition  matrix  is  a  simple  material  balance  relation  which 
is  obviously  auite  accurate.  Such  a  model  could  be  expected 
to  perform  ouite  well. 

B.   ESTIMATING  A  MEAN  FUNCTION 

AlthousJh  the  Kalman  filter  was  derived  from  an  assumption 
of  stationari ty y  we  have  seen  that  it  can  be  Quite  powerful 
in  separatinsi  a  time  varyinsJ  sisJnal  from  noise.  The  examples 
of  section  III  were  all  essentially  estimates  of  the 
time-v3ryin<3  mean  function  of  a  stochastic  process.  The 
example  process  was  Gaussian  with  a  constant  variance.  The 
variance  was  the  measurement  noise y  and  so  directly 
influenced  the  ^ain.  If  variance  were  not  constant?  the 
performance  of  a  non-adaptive  filter  would  be  dei^raded.  If 
the  change  in  variance  was  jareat  erioui-ihr  an  adaptive  filter 
would  be  rewuired. 

A  dood  method  of  estimatin:^  a  time-varying  mean  function 
could  be  applied  in  numerous  areas?  such  as  any  sort  of 
traffic  or  flow  control  problem?  perhaps  in  Quality  control 
of  larde-batch  or  flow  manufacturing  processes?  and  any 
application  where  it   is  desirable  to  detect  a   chanise  in  the 
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process.  The  sensitivity  of  the  filter  is  directly 
adjustable  bvj  the  modeller  throu:^h  the  noise  covarisnce 
matrices  Q  and  R. 

A  particularly  useful  application  would  he  to  the 
estimation  of  the  rate  parameter  of  a  non-time-homodeneous 
Poisson  process*  If  this  can  be  done  accurately y  the  process 
can  be  transformed  to  a  stationary  one  Cref»6Il?  which  i^^reatly 
expands  the  number  of  analytic  tools  that  can  be  used* 

The  Poisson  process  is  a  counting  process  in  continuous 
times'  and  to  attempt  to  filter  a  strinsi  of  interarrival  time 
data  would  violate  the  sampling  theorem.  The  times  of 
arrival  are  the  measurement  times y  and  they  are  most 
certainly  not  made  at  eaually  spaced  intervals.  Insteady  the 
filter  may  be  desiidned  to  sample  a  countin?^  process.  At 
discrete  intervals  the  filter  would  count  the  number  of 
arrivals  since  some  arbitrary  time  oriri^in.  If  the  process 
were  to  continue  for  a  lon:^  timey  the  time  origin  misiht 
occasionally  have  to  be  reset  to  prevent  computer  overflow. 
It  is  easy  to  see  how  this  samplin.<^  process  could  be 
implemented  even  if  the  input  data  were  actually  arrival 
instants  in  continuous  time.  The  sampling  interval  should  be 
small  enough  that  there  is  low  probability  that  more  than  one 
arrival  would  occur  durind  a  sJiven  measurement  interval. 
Since  the  number  of  arrivals  is  monotone  non-decreasini-:J  in 
timey  a  velocity  or  trend  model  would  be  appropriate.  The 
input  data  would  consist  of  intej^ers.  The  state  estimates 
would  not.  The  non-inteder  estimate  of  number  of  arrivals  up 
to  the  current  time  would  not   be  useful  to  us.   However y  the 
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second  element  of  the  state  vector^  the  velocity  or  trends' 
would  in  fact  be  the  filtered  arrival  rate  estimate ♦  Since 
the  process  is  noisy  and  non-Gaussian?  a  very  low 
steady-state  dain  is  appropriate* 

The  time-varyind  Poisson  process  cannot  have  constant 
variance f  since  the  mean  and  the  variance  are  eQU3l4  A  low 
arrival  rate  implies  hisih  variance  in  the  Poisson  process  y 
which  is  eauivalent  to  hij^h  measurement  noisey  which  reauires 
low  ^ain*  A  const3nt-:3ain  filter  would  therefore  be 
relatively  more  sensitive  at  low  arrival  rates  than  at  hi:3h 
arrival  rates*  An  adaptive  filter  could  be  easily  desij^ned 
to  use  the  inverse  of  the  rate  estimate  as  the  measurement 
noise  variance  estimate*  Stability  misJht  reauire  that  the 
adjustment  of  the  measurement  noise  variance  be  itself  a 
filtering  process^  in  which  the  incominsJ  variance  estimate  is 
regarded  as  data* 

C»   MULTIPLE  REGRESSION 

If  the  resiression  constants  are  assumed  known  (or 
computed  by  other  means)  the  design  of  an  appropriate  filter 
is  Quite  straightforward)'  and  quality  of  estimation  is 
related  directly  to  the  Quality  of  the  model.  Note  that  the 
velocity  filter  is  simply  the  regression  of  velocity  on 
position?  where  the  slope  parameter  is  known  to  be  one*  If 
the  reslression  coefficients  are  assumed  to  vary  in  time?  the 
problem  becomes  non~ linear  and  is  auite  complex*  Because  of 
the    immense   applicability    of    this   model?    additional 
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developmental   work  is   indeed   3   fertile  field   for   future 
research ♦ 

D.   SOME  DESIGN  CONSIDERATIONS 

In  applications  where  the  noise  covariance  matrices  R  and 
Qr  and  the  system  dynamic  model  (state  transition  matrix  ^) 
are  known  or  easily  estimated)'  desisJn  is  straightforward 
and  has  been  successfully  accomplished  while  remainini^  in  the 
time  domain*  However  r  in  applications  where  sweepinsJ 
assumptions  are  reauired?  a  freauency -domain  analysis  could 
he  very  helpful*  Some  sjuidelines  are  as  followsJ 

1 .   Spectral  Analysis  of    the  Data 

A  spectral  analysis  of  sample  data  will  show  what  the 
freauency  response  of  the  filter  should  he*  The  Fast  Fourier 
Transform  (FFT)  prc^ram  available  in  most  computer  libraries 
is  Generally  easy  to  use.  Howevery  the  FFT  pr-o-^rams 
Generally  reauire  an  exact  power  of  2  for  the  number  of  data 
points*  H3mmin.<3  Cref.7;j  points  out  some  pitfalls*  Since 
stationarity  is  assumed^  the  data  should  be  considered  as  a 
rotating  cylinder^  and  if  the  starting  and  endini^  values  are 
not  similarif  a  discontinuity  will  exist  in  the  spectrum*  The 
data  can  be  tapered  and  padded  with  zeros»  but  exactly  the 
best  method  to  accomplish  this  is  unknown.  Several  methods 
mi^ht  be  tried* 

The  main  virtue  of  the  FFT   is  its  speed*   It  works  well 
on  a  lond  run  of  data*   If  the  number  of  data  points  is  small 
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(around  a  hundred)  it  mii^ht  be  effective  to  find  (or  write)  a 
less  efficient.'  conventional  discrete  Fourier  transform 
program!'  which  would  not  reauire  padded  y  truncated?  or 
tapered  data  if  the  starting  and  ending  values  are  similar* 
If  the  FFT  program  used  does  not  reauire  an  exact  power  of 
two  for  the  number  of  input  data  points?  it  would  be  well  to 
find  out  why  not*  The  program  may  be  doins^  the  paddin<3  and 
tsperind  itselfir  and  the  analyst  should  be  curious  as  to  how* 
The  analyst  should  remember  that  the  spectrum  is  computed 
from  the  dstay  and  it  is  therefore  an  estimate*  If  the  run 
of  data  is  shorts  there  will  be  considerable  variance  in  the 
estimate ♦ 

2*   Frequency  Analysis  of  Proposed  Models 

The  analyst  may  test  the  effect  of  assumptions  made 
in  desidnind  the  filter  by  simply  obtaining  an  impulse 
response  of  the  filter  and  running  it  throu^^h  an  FFT. 
Truncation  and   t3Perin<3  is  no   problem y  because   the  impulse 

response  will   approach  zero   with  time*   The  profer       impulse 

n 
function  is  simply  a   1  followed  by  2  -1  neros   for  a  filtery 

or  a  1   in  the  middle  of  2   -1  zeros  for  a   smoother*   If  the 

output  of  the  FFT  consists   of  real  and  imaj^inary  componentsy 

it  will  be  necessary  to  compute  amplitude  and  phase* 

3*   Adjusting  the  Model 

If  the  model  dynamics  seem  adequate  but  the  bandwidth 
is  wrondy  the  analyst  should  by  now  have  some  insight  into 
what  adjustments  to  make  to   the  noise  covariance  matrices  to 
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try  to  iiTiPv^ove  things*  In  a  model  of  any  coiriplexity  at  all? 
there  are  numerous  passible  conibinations  ♦  However!'  even  soine 
improvement  over  the  initial  assumptions  will  be  beneficial* 
Ule  are  not  looking  for  theoretical  elegance*  we  avQ  lookinsJ 
for  performance* 

Perhaps  the  model  dynamics  obviously  call  for  a  trend 
filter  or  even  a  ch3n.^e-of -trend  (acceleration)  filterr  but 
the  data   is  auite  noisy*    Consideration  should  be   !:;5iven  to 

lowerinsJ  the  order  of  the  filter*  A  very  low-^ain  velocity 
filter  will  not  follow  changes  in  trend  well*  A  hi^her-:3ain 
scalar  filter  may  do  so  more  ef  f  ecti  vely  ?  3lthou<3h  it  will 
lad  a  steady  trend*  There  are  many  tradeoffs^  and  we  cannot 
achieve  perfection. 

4 ♦   Testing  the  Model 

The  model  should  be  tested  on  real  or  simulated  data* 
From  here  onr  the  model  lin^:^  process  is  the  standard  cyclical 
oney  doings  back  to  earlier  steps  as  necessary  until 
satisfactory  performance  is  achieved* 
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APPENDIX  A>   DEPg NATIONS 

1*   SCALAR  KALMAN  FILTER 

3*   Recursive  Forinula  for  Kalman  Gain 

The  cov3ri<3rice  extrapolation  eauation 
P<t)  =  I  Z(t-l)  -^  i    Q 
reduces  in  the  scalar  case  to 

P<t)  =  2<t-l)  +  Q 
Since 

K(t)  =  2(t)  H^r"' 
we  may  write?  for  the  scalar  case? 

2(t-l)  =  K(t-l)  R 

Similarly?  since 

T      r     -1 

K(t)  =  PH  C  HPH  +  R3 

by  reducing  to  the  scalar  case  and  substituting!-  we  may  write 

K(t)  =    K(t-l)  R  -f  Q    =    K(t-1)  +  Q/R 

K(t-l)  R  +  a  f  R    K(t-l)  f  Q/R  +  1 

b .   Steady-State  Kalman  Gain 

Rearran^in^^  the  recursive  slain  equation  and  letting 
K(t)  =  K(t-l)  =  K 
we  see  that 

K^f  (a/R)K  -  Q/R  =  0 
By  the  Quadratic  formula? 


2R     /  4R*   R 


We  are   obliged  to   take  the  larder   root?  since   the  smaller 
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root  would  force  the  sJsin  to  be  negative.  We  also  observe  the 
inverse  relationshiF' 

(K  +  a/2R)^  =  a^/4R*+  Q/R 
K^+  (a/R)K  +  a^/4R^  •■=  G*/4R*+  Q/R 
K*  =  (Q/R)(l-K) 
Q/R  =  K*/(.l-K) 

c  ♦   Transient  Kaltrian  Gain 

Recall  that  the  K'alman  filter  reauires  a  prior  state 
estimate  X(0)  and  a  prior  estimate  of  covariance  F'(0).  This 
reauirement  can  be  avoided  by  usinsJ  K(l)  =  1?  which  allows 
the  initial  state  estimate  to  be  eaual  to  the  first 
measurement*   Recall  that 

K(t)  =  I(t)  H  R 


Since  K(l)  =  1?    then  2(1)  ■■=    R 


d*   Amplitude  and  Phase  of  FreGuency  Response 


The  freauency  response  is 


H(v)  =  3  2-Cb  e>cp(~iv)3^ 


sxnce 


then 


|b  exp(-iv) I  <  1 


H(v)  =  a  /  lll-b  exp(-iv)3 

The  amplitude  squared  is 

2  2 

A   =  H(v)  H(-v)  ••=  a  /  Cl-b  exp(-iv):]  Cl-b  eKP(iv)3 


9ii 


B  y    hi  I.  J 1  e  r  '  s    r  e  1  <3 1  i  o  ri 

A^     ^:=    a^/    CI     f    tf    ■-2b    cos    vl 
which    iTisy    be    writto^n 

A*    -■--    3^/    C(l-b)*f    2b (1 -cos    v).] 


recsl  1  iriid    that 


Q/R    ^    K^/(l-K)    =    3^/b 


WG    may    write    the    arriplitude    as 


a/R 


Q/R    f    2(l-cos    V) 


The    phase    an^^le    is 

9(v)    =    arctan    ClmCv)    /    Re(v)3 
where    Iiti(v)       and    Re(v)       are    the    ima.<3inary       and    real       parts    of 
H(v)»    which    iTiay    be    written 


H  ( V ) 


II 1  •-  b    e  ;^  p  (  i  V  )  1 


lll-ta    exp(-iv)  JCl-b    exp<iv)3 

H  (  V )    =    a    ( 1 f  b    c Q s    V    - i b    si n    v ) 
J.     f    b        •■2b    cos    V 

which    allows    us    to    write 

8(v)    ~    arctan    C ( -b    sin    v)    /    ( 1 ~b    cos    v ) 3 

The    an^le    -for    niaxiiTtum    phase    shift    occurs    when 


d8(v)    -      b    --    b    cos    V 


0 


dv  1     f    b*  -2b    cos    V 

so  tha-t  the  iTiaKimum  phase  shift  9(v){Tiax  occurs  when 

V  -  arccos  b 

and  has  a  value  of 

8(v)nriax  =  arctan   -b  sin  (arccos  b) 

1-b  cos (arccos  b) 


arctan  (  ~b  /  ./1-b'-  ) 


/ 
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2»  IMPULSE  RESPONSE  FUNCTIONS 


The  .i(TiPul<j;e- response  of  the  scalar  Kalman  filter  is 


■t 
d<t)=3b       t=0ylr27.». 


a*       IiTiPulse-response    of    Double    Filter 


^(t)    =    ab  0  3b 
^  t 


t )     -     ^^  ah    ao  =    a     ^^  b    b 


d(t)    -■=    (t  +  1)    3^b^ 


b*     Inripulse-response    of    Scalar    Kalman    Smoother 


^(t)    =    ab     ®  ab 


£i  ( t )    -     >^_  a    b    a  b         -a  ^  b         =    a    b     ^^  b 


<J(t)    ■•=    a^b^/    (1-b)       =    a    b*/(l+b) 
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